Perform experiments and continuously optimize your communication campaigns with A/B testing. Testing audiences in the Flow tool enables you to see how they react to your content, what channel they prefer, or what is the best time to send them communication.
You can do the following:
- Compare how different channels perform with your audience.
- Perform A/B testing for all available channels.
- Manually pretest different versions and choose the winner.
- To test variations in versions, change any parameter or attribute of a message for the different test versions.
- Compare the testing results for different versions.
Set Up an A/B Test
We will show an example with different channels involving similar content - a discount code that a company would like to communicate to its customers. After a few hours or days (depending on the timeframe specified for this communication), the metrics will show which channel would be the best choice in the future.
- In the Moments module, select an entry point for your communication (for this example case, we will use Item Added to Wishlist event as our communication trigger), and then add the Split Audience element - you can select how many audiences you want and split them in proportions of your choice. Separate branches for each subset audience will be created as seen in the image below, with people randomly being assigned to these branches.
- Select channels and messages that will best serve your desired testing scenario.
- Next, include the Evaluate Message element with keywords that will let you know the customer reaction. For Email, select link clicked instead of using the keyword.
- If the customers responded using the keyword, send another SMS and Viber message with instructions involving the promo code. Also, you can send a message to customers who clicked the link in the email message.
- Use an event such as Item Purchased. This event will let you know whether the customer has reacted to your message and completed the purchase. Think about how long you want to wait before sending another message to them based on their actions. For example, you could wait 2 days.
Finally, if you want, you can send a thank you message to those who made a purchase. You could also decide to continue the Flow by sending another reminder to those who have not reacted to your communication.
After you Validate and Launch the Flow, you can go back to it and check the Flow Performance.
The outcome of the A/B Testing shown in this article offers a couple of possibilities, including the continuation of the same campaign or planning future communication with customers.
Use Performance Data For Future Campaigns
In the Moments module, click on your campaign to see how the channels performed. If, for instance, SMS was the most successful one, your next campaign could be focused on the SMS messages to customers.
The data you will see when hovering over each channel (in this example case SMS, Viber, and Email), involves the number of customers that the message was sent to, the total number of engagements that includes possible multiple entries to the Flow by some customers, and the total number of the sent messages.
Pretest and Send Winner Message
Another A/B Testing option in Flow is to add a tag to your communication. The tag would represent a winning segment (tag can have any name). The first three audience branches are used for pretesting, and the best performing message you can later send to the remaining audience branch via a new campaign that you would create in Flow. This we will explain in the following paragraphs.
New Campaign in Flow
Go back to the Moments page and create another flow same as the winning one or duplicate the existing one. But this time use the tag that involves the remaining audience.
- Select the Predefined Audience starting point and select tag winning_segment.
- Add the SMS message.
- Add Evaluate Inbound SMS.
- Add another message and a Wait for an event element; in the side panel, select Item Purchased event. Select the wait time 2 days.
- The final Flow would look something like this: