Quantifying incrementality (loyalty attribution) is a question I’ve wrestled with my whole career.  I remember when I was running the loyalty program for a hotel chain and a GM from one of the properties cornered me. “My property spends more than $10,000 a month giving points to loyalty members– how many of those guests would have stayed with without the points”

I did have an answer, because we routinely did several analyses to understand the incrementality of the program. Yes, several analyses.  Attributing customer behavior in a not a simple analysis and often not a singular one.

My last blog post accidental beneficiaries ate my lunch was about how non-incremental behavior can sink a loyalty program.  I received a number of comments that the math all made sense; but how do you assess if customer behavior is incremental? So in this post I’ll share some thinking and examples of how to assess if behavior in a loyalty program is incremental.

First, some best practices

In assessing and quantifying incrementality; four best practices we have found

  1. Define the use cases for incremental behavior. In any analytical effort, the first step is to define what it is being measured.  In assessing the incrementality of a loyalty program; our starting point is to define the “use cases” for how the loyalty program is expected to grow customer value. These use cases drive the metrics we want to create.
  2. In creating metrics, look beyond standard reports. Standard reports like enrollments, active members, and total member spending, measure if a loyalty program is popular, not if the loyalty program is effective. Assessing incremental behavior requires a more rigorous examination where specific metrics are built based on these “use cases” for incremental behavior.
  3. Understand the why and with whom At Sprocket we always try to employ “mixed method” approach with both quantitative analysis and also qualitative. Not only do we want to answer the question how much behavior is incremental, we also want to understand what aspects of the value proposition drive incremental behavior and which customers are more incremental and which are accidental beneficiaries.

Some analytical tools 

In wrestling with incrementality every match is a little different. There is no one singular way to attribute customer behavior.  I’ll share a few of examples of the analytical tools we employ to assess incremental behavior. (Warning this post is a bit long… but I included graphics).   

Member versus Non Member “Basket Analysis”
Use Cases:  Are we increasing spending with each trip?  Are members less dependent on discounts?

Imagine a restaurant that wants use their loyalty program to get members to add an appetizer and dessert, or bring a friend along.  Or perhaps the premise of the loyalty is to decrease the reliance on coupons and wean customers from being deal oriented.

An analysis to get these at use cases is a member versus non-member basket analysis—where member transactions are compared to non-members on various dimensions – average spend, items purchased, margin etc.

Figures 1 and 2

 

Transactions Distribution
Use Case:  Increase Frequency of Purchase

Just about every loyalty program I have ever been involved with has had the objective of getting customers through the door more often. Frequency of purchase can be visualized with a histogram (a transaction distribution).

Figure 3

Comparing the histograms of members to non-members provides insight as to if the program is driving more trips. Taking the increase in average trips for members times average spend per trip times the number of active members puts a dollar figure on the lift.

Comp Customer Analysis
Use Cases:  Are we growing customer value?  Are we shifting share?

 A fundamental metric most financial analysts use to assess business performance metric is same store sales, (or comparable store sales). The same metric can be generated on a per customer basis.  At Sprocket we like to call this a Comp Customer Analysis. Another term is “up trender / down trender”

While a little more complicated an analysis, a “comp customer analysis”  can be a powerful way to quantify incrementality.

An up trending member is one who has more transactions in the last 12 months than he/she did in the previous 12 months.  A down trending member is one who has fewer transactions in the last 12 months than he/she did in the previous 12 months.

To create an up trender analysis an extract of all members who have been active for 24 months needs to be generated. This allows a year over “delta” for each member to be calculated.

Figure 4

A histogram visually portrays the number of up trenders vs. down trenders:

Figure 5

In the above example, the program is “winning” with slightly more customers than it is “losing” with.

Calculating the “delta” for associated spending quantifies the comp customer analysis in dollar terms.

Tracking up trender, down trender over time shows if the program is sustaining comp customer growth.  In the above example $1.3MM in comp customer growth in 2016 was followed by $1.7MM customer growth in 2017.

Treatment vs Control
Use Cases:  Was a promotion effective? 

A common promotion in loyalty programs is thresh-hold promotion.  For example a hotel program may have a summer mega bonus for members who stay 3 times between July 1 and September 30.

While promotions like this are communicated to the entire membership; qualifiers versus non-qualifiers become a quasi treatment versus control group.  This sets up analysis as shown below.

Figure 7

Visualizing the data demonstrates the lift the promotion delivered.

Understand the why and with whom

At Sprocket we always try to employ a mixed method approach with our analysis.  When wrestling with incrementality combining qualitative with the quantitative analysis is particularly helpful.

Quantitative analysis can put a number of how much of the member behavior is incremental. The clever use of qualitative analysis can help understand what aspects of the value proposition are driving that behavior.

For example, loyalty programs often survey members about how important different aspects of the value proposition are.  By overlaying survey responses with an up trender / down trender analysis (Figure 7), we can gain an understanding of what aspects of the value proposition are driving that incremental behavior.

When surveying program members, we often ask a question specifically around incrementality.  For example:

Thinking about how much you stayed at  ______________ over the last year, the frequent guest program influenced me to stay:

  • A GREAT DEAL more than I otherwise would have
  • A FEW TIMES more than I otherwise would have
  • 1 or 2 TIMES more than I otherwise would have
  • The program had NO INFLUENCE

This survey response, when paired with behaviors and personas, can provide a good understanding of which types of customers the program drives incremental behavior with.

Figure 10a

Figure 10b

 

While not easy, it’s necessary and doable

Every loyalty program manager should be wrestling incrementality.  But be ready for a tough battle.  The questions are formidable. How much of the behavior in the program is incremental?  What’s that behavior worth?  Which members exhibit incremental behavior and what motivates them?

While not an easy pin fall— wrestling with incremenatly is a winnable match. By combing clever analysis tied to the use cases for incremental behavior with a mixed method approach using both quantitative and qualitative analysis, incrementality can be wrestled to the mat.