by Lilia | May 12, 2023 | Announcements, Email Marketing, Features
Many of our customers have been waiting for AB testing and it’s finally live and can be used on our free plan. You can use subject line AB testing on Bulk/Marketing Campaign type only, not on automations or transactional campaigns.
What Is an A/B Test Campaign?
An A/B test campaign is a campaign with 2 or more versions being tested on one dimension – typically subject line or send time. A multi-variate test is a test with more than 1 dimension, for example if you are testing 2 different subject lines with 2 different versions of your message body you have 4 total versions and a multi-variate test, to determine the best combination of subject line and message body.
Why Use AB Tests?
AB tests can help you optimize your campaign engagement, by determining best time or subject line to use. If you have a large list (over 50k) you can test a subject line on 20k and then use the best one on the rest of your list, improving your engagement overall.
You can setup AB test to determine:
- Ideal send time for your brand or list – test early on, then use that knowledge forever!
- Best subject line
- Best converting CTA in your email body
Besides producing better results (sales, conversions), higher engagement helps improve your future inboxing.
How to Setup an A/B or A/B/C Test on a Subject Line
Create your bulk/marketing campaign the same way you usually do, for example by copying and modifying your older campaign.
Determine how many variations do you want to test and click on the A/B Test button next to the subject line label:
AB testing is only available on bulk campaigns, not on transactional or automations. You can test up to 3 subject lines on a single campaign
TIP: You would need a good list size or high engagement to get statistically significant results.
A winner is selected at least 2 hours (or later) after campaign is sent, to allow for meaningful data collection, so you will only see raw data for the first 2 hours after a campaign is sent.
The winner is chosen based on open rate.
That’s it. If you are a BigMailer customer you can reach out for assistance with this or anything else via chat 7 days a week.
If you want to AB test on send time, please refer to this article for detailed instructions.
Not a BigMailer customer? Why not try BigMailer for free, with no features locked? We are sure you will love it.
by Lilia | Feb 20, 2023 | Announcements, Email Marketing
Many customers who sent campaigns on Feb 20 reported high complaint rates. Upon further review of engagement by domain reports we are able to confirm that the complaint rates went up significantly for these email domains – hotmail.com, hotmail.co.uk, msn.com, live.com, live.co.uk
Why did this happen?
It appears there was a change in the Hotmail algorithm that determines what email go to spam folder vs. inbox. As a result, 3 things happened:
- A very high volume of spam is been seen in the Inbox.
- Regular senders inboxed more of their emails than they did historically, seeing higher open rates and as a result higher complaint rates.
- Some subscribers had to mass move emails to Spam folder or marked large batches of emails as Spam, likely marking some emails as spam without high intent.
What should you do?
Consider using segmentation and exclude high complaining domains in the next 1-2 days to lower your overall complaint rates, whether you are using Amazon SES and worried about suspension or just want to avoid getting caught in the crossfires.
As always, to keep complaint rate low, follow these email deliverability best practices:
- Make your unsubscribe link east to see/find, don’t make the text small
- Consider lowering your frequency if your complaint rates are high.
To ensure you maintain high deliverability and inboxing follow best practices. If you think you already do, please review this article on advanced deliverability and inboxing tips.
If you are a BigMailer customer and need assistance or would like an audit of your account please reach out to us via chat.
We will be posting updates here if you want to know when this issue gets resolved.
by Lilia | Feb 12, 2023 | Email Marketing, Guides
UPDATE: Subject line AB testing is now live. You can use this article to setup other type of AB tests, for example on send time.
What Is an A/B Test Campaign?
An A/B test campaign is a campaign with 2 or more versions being tested on one dimension – typically subject line or send time. A multi-variate test is a test with more than 1 dimension, for example if you are testing 2 different subject lines with 2 different versions of your message body you have 4 total versions and a multi-variate test, to determine the best combination of subject line and message body.
Why Use AB Tests?
AB tests can help you optimize your campaign engagement, by determining best time or subject line to use. If you have a large list (over 50k) you can test a subject line on 20k and use the best one on the rest of your list, improving your engagement overall.
You can setup AB test to determine:
- Ideal send time for your brand or list – test early on, then use that knowledge forever!
- Best subject line
- Best converting CTA in your email body
- A combination or all of the above!
Besides producing better results (sales, conversions), higher engagement helps improve your future inboxing.
How to Setup an A/B or A/B/C Test
Step 1: Setup up your A/B or A/B/C segments
Determine how many variations do you want to test – that’s how many segments you need to split your list into. We suggest to define reusable segments using “starts with” condition on one of your custom fields, for example first name.
Then, create a segment that’s basically “not A” to ensure you don’t make a mistake with rule overlap. The example below shows a segment C that’s basically doesn’t match ether A or B, and segment B would be setup similar to A but with different rules.
TIP: Make sure each segment has at least 10,000 contacts in it, so your experiment results are statistically significant.
Step 2: Setup your campaign variations.
Create your bulk/promo campaign, then create copies with your variation – subject line, preview, or message body. Assign a different segment to each campaign, see example in the screenshot below.
If you are testing on send time, then your campaigns will be identical, but will just be scheduled to send at different times.
Note: The example above is for a B2C list that’s 60%+ Gmail, so those landed in Promotions tab.
TIP: You could create a test using small parts of a larger list, then use the results for the remainder of your list.
Step 3: Start Testing (almost)
Reach out via chat and request to update your account to allow you to send more than 1 campaign at a time, if you want to test on content and send your campaigns at the same time. By default, accounts have 1 queue and campaigns go out sequentially. If you are on an Agency plan, you already have ability to send concurrent campaigns.
That’s it. If you are a BigMailer customer you can reach out for assistance with this or anything else via chat 7 days a week.
Not a BigMailer customer? Why not try BigMailer for free, with no features locked? We are sure you will love it.
