Scope

How to do targeted offer tests within the audience in Retention?

How to test an offer for a specific audience in Retention?

Targeted Offer Tests within Audiences in Retention

Summary

The rules on the Targeting page get evaluated from the top first.  The issue can arise when you have placed the rule for the new Audience as last on the targeting page. In order to resolve the issue, please move the required rule above the targeting page. Refer to the steps below on how targetted offer works within Audience.

A common use case for testing within Retention is presenting a random set of offers that are contextual to a specific audience. A few good examples are:

• Different Plan Types / Products
• Weekly v. Monthly v. Annual Billing Cadences
• Different Languages or Regions
• High / Low LTV customers

Solution

In each of these examples above, it may make sense for your business to have a subset of offers you only expose to this specific Audience vs the entire cancel population. Pauses won't make sense for most annual customers but are great for Monthly customers for example whereas you may want to show a stronger % discount to High LTV customers than you show to Low LTV customers. This is a great opportunity to create targeted Offer tests within Audiences. 

The steps to set these tests up are very similar to the Random Offer Test, except instead of creating one set of Offers / Pages, we create one for each Audience we choose to target. For this example, we are going to set up a simple Monthly v. Annual Audience split and test different offers within them. We will take the following steps:

1. Define your Audiences in the Audience Manager
2. Create your Default Page as your Holdout
3. Create 2 to 4 Different Offer Types for each Audience
4. Create 2 to 4 Matching Pages and Assign Offers
5. Optional: Create Loss Aversion Offer
6. Configure your Targeting Tree to set up the Test
7. Set a Fallback & Publish

Step 1: Define your audiences in the audience manager

Refer to this article to set up an Audience. For the purposes of this test, we will create a Monthly and an Annual Audience. 

Step 2: Create your default page as your holdout

Refer to this article to learn about updating pages in Retention. When creating your control page you will want to make sure that you are creating a generic page that generally represents your existing in-house cancel flow. This will help us monitor performance as the tests evolve and ensure we are not biased toward the initial test control with too many improvements. When you have multiple Audiences, it may make sense to have control per audience. This applies if your page has different elements on it for the various canceled Audiences. In this case (monthly v. annual) the offers will change but the page will not. Therefore we will create a single control page that we will target within each of the two Audiences.

Step 3: Create 2 to 4 different Offer Types

Once you have a control page, it is time to navigate to the Chargebee Retention Offer Library and create three to four different Offer Types. 

Targeting multiple Audiences lets you set up different sets of Offers per audience. This makes sense in many cases because the incentives you are willing to give up may be contextual to that specific Audience. In our example case, we don't want to show skip offers to annual customers and likewise don't want to show downgrade to monthly to a monthly customer. Therefore we set up two different sets of offers, one for monthly and one for annual. Our offer mix will look like this:

Annual

• Plan Change 
• 25% Discount
• Downgrade to Monthly

Monthly

• Upgrade Offer Test
• Skip Offer Test

Step 4: Create 2 to 4 matching pages and assign offers

After creating your offers, you should navigate back to the pages tab. Here we will create 5 pages, one for each of the offers in our annual and monthly audience.  

In this case, your pages tab would look something like the above. Now you are ready to go add these audience splits in the targeting tab to set up your test. 

(Optional) Step 5: Create Loss Aversion offer

One of the immediate opportunities to generate lift in Chargebee Retention comes by layering two offers within a single Page. This allows you to present an initial offer (say a Skip) in the Loss Aversion card that everyone on the page sees. Then when they select a reason, they will be presented with a second often stronger offer such as a discount of full pause. Several of our customers leverage this by applying a Loss Aversion offer right from the beginning of their initial test. 

To create a Loss Aversion Offer, navigate to the offer library to create a new Loss Aversion offer. Select the goal you would like, in this case, we will choose a Skip. Under the layout section scroll to the bottom and select the Loss Aversion CTA Card. 

Once you have created this card, you will place it in the Card B slot for all of your pages except for the control. Setting this test up with a Loss Aversion card on each page beside the control allows us to isolate the impact of the Loss Aversion offer against the control, without distorting the random offer test by showing the same Loss Aversion offer to every customer that sees a random offer.

Step 6: Add your audience splits to the test

Now it is time to navigate to the Targeting tab and configure the routing for your test. Select the '+ Target Audience' button and add your first Audience to the Tree. 

Note we will prioritize the Audience logic from top to bottom, so put your highest priority audiences first.
From there you will add each of your relevant pages under the monthly and annual audiences. In this case, we will set a control within each audience vs. at the top level. This allows us to report on the control per audience vs. having it separated at the top.

Step 7: Set a fallback & publish

Whenever you have Audiences defined in a test you need to set a fallback Page in Retention that will show if the fields used to define the Audience are not present. In this case, we will simply set the control as the fallback and publish our new test.

Once your test is live, you will be able to use the Experience Performance Report to understand how each page is performing.