Optimize communication through A/B testing

A/B testing in an experiment that enables you to compare different variations of a communication to identify which one performs better. Compare the results to see how your audiences react to your content, what channel they prefer, or the best time to send them communication. Use these results to optimize your communication campaigns.

The following are examples of how to use A/B testing.

Compare how different channels perform with your audience.
Pretest different versions of a campaign and identify the one that performs the best.

This tutorial explains how to set up an A/B test for different channels within a flow and identify the channel that performs the best.

You can use any supported channel. This tutorial uses SMS, Apple Messages for Business, and Email channels.

Products and channels

Prerequisites

Infobip account with Moments enabled. If you do not have an account, sign up (opens in a new tab) for an account.
At least two supported channels.

Process overview

In a flow campaign, split your audience into 2 or more groups.
Send each group the same message but through different channels.
See the metrics to identify which channel performed the best.
Use these metrics to optimize the existing campaign or plan a future campaign.

Implementation steps

In a flow in Moments, select an entry point for the communication. This example uses the People real-time event entry point with the Item added to wishlist event as the trigger.
Add the Split Audience element. Select the number of subsets into which you want to divide the audience, and the proportion for each subset.

Separate branches are created for each audience. End users are randomly assigned to each branch.
Add different channels with a similar message in each channel. This example uses SMS, Apple Messages for Business, and Email channels. The message offers a 30% discount to the end users.
For all channels except Email: Add the Evaluate inbound message element. In the channel elements, add keywords that help you know the end user's reaction.

For Email: Add the Wait for an event element and select Email link clicked.
For all channels except Email: If the end users responded using the keyword, send another message with instructions about the discount.

For Email: If the end users clicked the link, send another message with instructions about the discount.
To each channel, add the Wait for an event element and select an event (opens in a new tab) such as Item purchased. This event lets you know whether the end user reacted to your message and completed the purchase. Configure how long you want to wait before sending another message based on their actions. Example: 2 days.
(Optional) Send a thank you message to those who made a purchase. You could also decide to continue the flow by sending another reminder to those who have not reacted to your communication.
After you validate and launch the flow, you can check the flow performance.

Analytics

Select your campaign to see how the channels performed. Hover over a channel element to view the statistics for that channel.

The statistics are channel specific. The Number of engaged persons and Performance ratio information is available for all channels.

Example: For the SMS channel, you can view the following data.

Number of end users to whom the message was sent.
Total number of engagements, which includes multiple entries into the flow by some end users.
Total number of sent messages.

Next steps

Use the results of the A/B test to optimize your campaign or to plan future communication with end users.

Use performance data for future campaigns

Use the performance data to build future campaigns. Example: If SMS was the most successful channel, use the SMS channel to send messages to end users in your next campaign.

Additional options

The following are other ways in which you can use A/B testing.

Pretest and send the best message

Another A/B testing option in Flow is to test messages for a segment of your audience and send the best message to the remaining audience.

The steps are similar to the ones in the Implementation steps section. In this case, split the audience into 4. Use 3 audience branches for pretesting. Add a tag to the fourth segment. Example: winning_segment.

After you identify the best performing message among the 3 audience segments, you can send that message to the fourth segment through a new campaign. Do the following.

In Moments, either create another flow that is similar to the earlier one or duplicate the flow.
Select the Predefined Audience entry point.
You need to select the fourth segment, which you created in the earlier flow, as the audience. You can identify this segment from the tag that you added to it. So, add the winning_segment tag to the entry point.
Add a Send SMS message element. The message offers a 30% discount to the end users.
Add the Evaluate Inbound SMS element. In the element, add keywords that help you know the end user's reaction.
If the end users responded using the keyword, send another message with instructions about the discount. Add a Send SMS message element.
Add the Wait for an event element and select an event such as Item purchased. This event lets you know whether the end user reacted to your message and completed the purchase. Configure how long you want to wait before sending another message based on their actions. Example: 2 days.

The flow should be similar to the following image.

Compare multiple flow versions

To test variations in flow versions, change any parameter or attribute of a message for the different test versions. Then, compare the testing results for different versions.