Sign in or start a free trial to avail of this feature.
6. ETS Analysis and Comparing Models
In this lesson we will learn how to deploy a second time series model to the workflow and contrast the results with the alternative in order to determine the more appropriate model for our dataset.
- This tool estimates a time series forecasting model using an exponential smoothing method
- Outputs of this tool are similar to that of the ARIMA tool
- This tool compares one or more time series models and measures model accuracy
- Any models that are being compared must be joined together with a union tool before being connected tool
- The validation dataset must also be connected to this tool
- Comparing forecast errors for the models can give us a good idea of which model is the best fit for our forecast
In our previous lesson, we partitioned our data, using the Create Samples tool and ran 70% of our data through the ARIMA model, yielding our first forecast, this was just the first step in our overall goal of generating a sales forecast for the upcoming year. In this lesson, we'll determine whether the ETS or ARIMA model is the preferred model for our forecast, to accomplish this goal, we'll apply the following steps.
First, we'll run our estimation set through an ETS model, next we'll analyze the results from the ETS model and compare it with the ARIMA model, finally we'll compare both models with a validation set and make the final decision on which to use for our forecast. For our first step, we'll run an ETS analysis on our estimation set, to that end, we'll navigate to the Time Series tab and connect an ETS tool to the estimation set, we'll name the model Sales_ETS, targeting Weekly_Sales with the Weekly frequency, again we'll navigate to the Other options tab, select the Series starting period box, enter the year 2014 and then choose 52 periods for the forecast plot, we'll Add All Browses and run the Workflow.
We'll now move on to the second step and analyze the results from the ETS model, we'll then compare it with the results from the ARIMA model. Let's start by reviewing the Browse icon connected to the R Report output node, we can see in analysis of our ETS model, that includes a Decomposition of our data, which details various trends, cycles and seasonality, a quick look at the Forecast graph shows that the best forecast is a simple average of the data, at first glance, this is not overly helpful, again the sample error measures are detailed. Let's move on to the interactive output report, we can see that the data from the report output is regenerated here with another graph of our forecast output, including confidence levels. Note that the forecast for the ETS model is an unsatisfyingly straight line, if we look at the ARIMA model again, we can see that the forecast is much more dynamic, based on these Forecast graphs, it looks like the ARIMA model may be more useful for our data. At this point, we'll move on to our final step and compare both models against the validation set using the TS Compare tool, we'll then draw out final conclusions. Before we add the TS Compare tool, we'll need to bring both models together with a Union join, we'll bring a Union tool onto the canvas and connect the output nodes from both the ARIMA model and the ETS model to the union input, we'll now navigate to the Time Series tab and bring a TS Compare tool onto the canvas, connecting the union output to the left input node, we can now apply these model configurations to our validation dataset, using the TS Compare tool, remember the validation set was created using the Create Samples tool and is available from the V validation output node, we'll connect this output to the R input node of the TS Compare tool, we'll then Add All Browsers and run the Workflow.
Before clicking on the Browse tools, we'll click on the TS Compare output node and consider the data presented in the Results window, we can see that the various forecast errors are contrasted, remember we're looking for the smallest errors, that is the ones closest to zero, it's quite clear that the ARIMA model is performing better across these indicators, if we look at the R report output node, we can see that it lists the forecast values for each model.
Moving on to the interactive report, we can see that it presents the same data graphically, from our analysis, we can conclude that the ARIMA model is a more appropriate technique for forecasting our sales data. Before moving on, let's put our Sample tool and all our connected tools into a container, we'll name this container Model Comparison and disable it. Let's quickly recap what we've done in this lesson, first we ran our estimation set through an ETS model, next we analyzed the results from the ETS model and compared them with the results from the ARIMA model, finally we compared both models to the validation set and determined that the ARIMA model is the most appropriate for our forecast. In our next lesson, we'll conclude our initial examination of the Time Series tools by applying the model to the entire dataset and then calculating the forecast values.