5. AB Trend and Controls Tools

 
Subtitles Enabled

Sign up for a free trial to access more free content.

Free trial

Overview

In this lesson, we will use the AB Trend tool to study the underlying trends and seasonality in the dataset, and then use the AB Controls tool to pair each treatment with one or more controls.

Lesson Notes

AB Trend Tool

  • The AB Trend tool identifies any underlying trends or seasonal patterns for each item in the dataset
  • The input for this tool should be the larger grouped dataset
  • The time period used for this tool must be at least one year plus the length of trial period
  • This means that uses must have historic data equal to at least one year plus the length of trial period
  • Note that this tool is often used as an input for the AB Controls tool

AB Controls Tool

  • The AB Controls tool matches the specified number of control units to each treatment unit as identified by the AB Treatments tool
  • This tool requires a numeric measure that upon which to match the treatment and control units
  • Users can specify anywhere between 1-20 control units for each treatment unit

Transcript

Over the past few lessons, we've successfully determined the ideal group of Ben's Beverages stores, to trial the revamped store layout. Our ultimate goal will be to compare the stores in our treatment group, with a control group. Based on this comparison, we can determine the effectiveness of the redesign project. In this lesson, our goal is to pair the stores from our treatment group with stores from the control group to prep for our analysis. We'll accomplish this goal by following two key steps.

First, we'll establish general underlying and seasonal trends on our weekly sales data for all stores. We'll then use an AB Controls Tool to identify our treatment and control pairs based on the trends.

We'll begin by using the AB Trend Tool to calculate the underlying trends in our data. This will help us pair appropriate control and treatment stores. We'll navigate to the AB Testing Tab on the Tools Palette and bring an AB Trend Tool onto the Canvas.

We'll connect it to the Summarize Tool, and the Historic Sample Groups Container. In the Configuration Window, we need to set the unit identifier to store, and the reporting period dates as weekly date.

Next, we must specify the performance measure we wish to use to calculate the underlying trend in our data. Sales would be a logical choice, however, the managing partner is particularly interested in footfall.

Our historic receipts data is a good proxy for footfall.

For the purposes of this analysis, we'll chose the sum underscore receipts field as our performance measure.

In real world conditions, you would probably want to test your results under both scenarios. We've aggregated our data to the weekly level, so the reporting period type is weekly. Next, we need to determine the number of periods to calculate the trend. The managing director wants to run the trial over a six week period.

Note the Alteryx AB trend tool requires data going back at least a year plus the number of periods in the analysis in order to establish general underlying trend and seasonality patterns.

Therefore, we'll need 58 weeks of data to run our trend analysis.

If we look at the results window, we can see that our data set runs from January 6th, 2014 through February 9th, 2015.

Exactly 58 weeks. This means that our trend projection will start on February 16, 2015.

Clearly, these dates have been lined up to facilitate our example.

In a real work scenario, you will need to tailor your data set so that you have sufficient additional data on top of the base 52 weeks to match the desired trial period. We'll go back to the Configuration Window of the AB Trend Tool, specify February 16th 2015 as the start date, and run the Workflow.

We're now ready to move on to the next step and create the store pairs. We can use the output from the AB Trend Tool to inform the next tool in our Workflow, AB Controls.

We'll navigate to the AB Testing Tab, and bring down an AB Controls Tool connecting the D Input Node to the AB Trend Tool.

We must connect the T Input to our summarized treatment group.

We'll make this connection wireless to keep things neat.

The AB Controls Tool calculates the distance between each of our treatment and control stores to determine the nearest neighbor. We'll end up with 20 treatment stores paired with 20 control stores. Let's configure this tool.

We'll identify the treatment unit as the field store coming from the treatment group container. We'll then identify the data measures to match treatments to the controls as the field store coming from the AB Trend Tool.

We'll specify to match controlled units using both trend and seasonality.

Finally, we have the option of returning more than one control unit.

We have 20 treatment stores and 104 total stores available. So, we'll match four controls to each treatment.

As a final step, we'll add a browse tool to the A Output Node of the AB Controls Tool and run the Workflow.

We'll open up the browse results in a new window, and see a chart which displays a distance between each of our controls and each of our treatments.

Most of the data is contained closely together, though it is interesting to note that the treatment stores 122 and 25 are somewhat outliers.

Ben's Beverages may want to investigate these outliers further. As we scroll down, we're presented with a table which details the four control stores matched with each of our 20 treatment stores. Let's stop the lesson here. In the next lesson, we'll bring in the results from the trial and prepare for our final analysis.