AB testing is now LIVE

AB testing is now LIVE

Many of our customers have been waiting for AB testing and it’s finally live and can be used on our free plan. You can use subject line AB testing on Bulk/Marketing Campaign type only, not on automations or transactional campaigns.

What Is an A/B Test Campaign?

An A/B test campaign is a campaign with 2 or more versions being tested on one dimension – typically subject line or send time. A multi-variate test is a test with more than 1 dimension, for example if you are testing 2 different subject lines with 2 different versions of your message body you have 4 total versions and a multi-variate test, to determine the best combination of subject line and message body.

Why Use AB Tests?

AB tests can help you optimize your campaign engagement, by determining best time or subject line to use. If you have a large list (over 50k) you can test a subject line on 20k and then use the best one on the rest of your list, improving your engagement overall.

You can setup AB test to determine:

  1. Ideal send time for your brand or list – test early on, then use that knowledge forever!
  2. Best subject line
  3. Best converting CTA in your email body

Besides producing better results (sales, conversions), higher engagement helps improve your future inboxing.

How to Setup an A/B or A/B/C Test on a Subject Line

Create your bulk/marketing campaign the same way you usually do, for example by copying and modifying your older campaign.

Determine how many variations do you want to test and click on the A/B Test button next to the subject line label:


AB testing is only available on bulk campaigns, not on transactional or automations. You can test up to 3 subject lines on a single campaign

TIP: You would need a good list size or high engagement to get statistically significant results.

A winner is selected at least 2 hours (or later) after campaign is sent, to allow for meaningful data collection, so you will only see raw data for the first 2 hours after a campaign is sent. 

AB test results

The winner is chosen based on open rate.

AB test results

That’s it. If you are a BigMailer customer you can reach out for assistance with this or anything else via chat 7 days a week.

If you want to AB test on send time, please refer to this article for detailed instructions.

Not a BigMailer customer? Why not try BigMailer for free, with no features locked? We are sure you will love it.


High complaint rate on Feb 20 for Hotmail, MSN, Live

High complaint rate on Feb 20 for Hotmail, MSN, Live

Many customers who sent campaigns on Feb 20 reported high complaint rates. Upon further review of engagement by domain reports we are able to confirm that the complaint rates went up significantly for these email domains – hotmail.com, hotmail.co.uk, msn.com, live.com, live.co.uk

Why did this happen?

It appears there was a change in the Hotmail algorithm that determines what email go to spam folder vs. inbox. As a result, 3 things happened:

  • A very high volume of spam is been seen in the Inbox.
  • Regular senders inboxed more of their emails than they did historically, seeing higher open rates and as a result higher complaint rates.
  • Some subscribers had to mass move emails to Spam folder or marked large batches of emails as Spam, likely marking some emails as spam without high intent.

What should you do?

Consider using segmentation and exclude high complaining domains in the next 1-2 days to lower your overall complaint rates, whether you are using Amazon SES and worried about suspension or just want to avoid getting caught in the crossfires.

As always, to keep complaint rate low, follow these email deliverability best practices:

  1. Make your unsubscribe link east to see/find, don’t make the text small
  2. Consider lowering your frequency if your complaint rates are high.

To ensure you maintain high deliverability and inboxing follow best practices. If you think you already do, please review this article on advanced deliverability and inboxing tips.

If you are a BigMailer customer and need assistance or would like an audit of your account please reach out to us via chat.

We will be posting updates here if you want to know when this issue gets resolved.

BigMailer Adds Transactional Email Support

Whew… we were a little behind the timeline we committed to on our product roadmap, but support for transactional emails is now live!. You can now add and manage your transactional emails, in addition to marketing (bulk) and automation campaigns (aka drip or email sequences) in BigMailer, included in all plans at no additional cost.

Using SendGrid for your transactional emails now? Here is a SendGrid vs. BigMailer comparison.

How to use BigMailer transactional email campaigns:

  1. Use BigMailer for standalone transactional emails triggered within your application and store/manage your transactional email templates on BigMailer platform – this allows marketers to tweak CTA and content of those emails without getting developers involved.
  2. Use your defined transactional emails as triggers in automated/drip sequences.
  3. Automatically feed your transactional email contacts into appropriate lists for use in your marketing/bulk campaign. This is a benefit of managing your transactional and marketing/bulk emails on the same platform.

EXAMPLE USE. Say you have an email collection form on your blog or site to get users to subscribe to your newsletter that goes into a list named “Newsletter Signup”. Once you setup a transactional email contacts to go into your “Regi” list or possibly “Paid Customers” you can segment your marketing campaigns based on those lists.

How to setup transactional email with BigMailer:

  1. Click on your username in site header (top right cornet), then select “Account” in the dropdown.
  2. Select “API” tab in the page navigation, you will see your API key 
  3. Select “Campaigns” in the site header navigation, then go to “Transactional” tab and click “Create Campaign button” 
  4. Define all the fields for a campaign, save.
  5. Refer to API documentation to add email triggers to your application.

We will be expanding our APIs and documentation in the coming weeks and months, but if there is some feature you are looking for please don’t hesitate to reach out.