A/B testing email messaging (aka “split testing”) is one of those things that gets big nods of approval but only a minority of people actually do.

Why?

In a survey of our blog readers, 42.9% of respondents told us they don’t know what to test. 19% felt that they lacked the tools.

That ends here.

A/B testing can help to significantly increase email engagement and sales effectiveness.

Below are 18 tips telling you exactly what to test, with specific tools to use to execute, so you can go out and experiment your way to higher open, reply, and click-through rates.

A/B test all outreachStart A/B testing your emails for freeTry Yesware Free

But First, 3 Rules To Remember When A/B Testing Emails

There are 3 questions you always want to ask yourself before testing:

1. What one thing am I testing?
2. Do I have enough data to know the result is meaningful?
3. What metric is the best measure of success?

1. What one thing am I testing?

To understand what’s really impacting your results, it’s best to test just one thing at a time.

For example, if you were comparing two different subject lines, you’d want to keep the body of the email the same. If one email outperformed the other, it might be hard to tell whether the difference was because of the variation in the subject line or what was written in the email itself.

2. Do I have enough data to know the result is meaningful?

Because you want to make sure your results are as close to statistically significant as possible. A/B testing is perfect for improving marketing or sales prospecting emails that are sent to many people, such as drip campaigns and multi-touch email campaigns. In our own tests at Yesware, we try to aim for a minimum of 100 emails (50 get A version, 50 get B version) as our goal.

You can use this test (the first yellow box) to see if your uplift (open, click, or reply—depending on what you’re testing for) has statistical significance. If it does, great! Go ahead and apply the winner to emails going forward (or until your next A/B test, anyway ).

3. What is the measure of success?

You have three options for email engagement metrics:

1. Open Rate — Which email variation was opened by the most people?

2. Reply Rate — Which email variation did more people reply to?

3. Click-Through Rate — Which email variation was more effective at getting people to click on linked content?

Sometimes, your key metric will be obvious. If you change your call to action, you’re likely hoping to improve your reply rate.

Other A/B email tests could hypothetically apply to all three metrics. When testing the best time to send, for example, how would you measure success? Should it be the email that receives the most opens, replies, or overall clicks?

Fortunately, there are no right or wrong answers here. Just be sure to pick one and be consistent in measuring against this metric in all your tests.

Now on to the fun part — ideas!

Stay on top of your metricsDaily activity, engagement data, and outcomesTry Yesware Free

A/B Testing Email Ideas To Improve Your Open Rates

People open emails because the subject line and/or preview text appeal to them. You can A/B test variations of each to find what works best with your recipients.

Test #1: Send At Different Times or Days

We want you to work smarter right from your inbox. That means sending emails that arrive at the right time for your recipient.

Our Best Time to Send Email Map lays out a recommended send time based on your recipient’s location.

That way, you send on their terms (and when they are most likely to respond).

All you have to do is:

1. Enter your location
2. Input your recipient’s location
3. See the recommended time to send

In this AB test, we’d recommend sending at the top time versus another time of day.

Use this tool to schedule your emails to send when they want — so you can keep writing them when you want. 

Test #2: Statements vs Questions In Your Subject Line

Obama’s best performing email subject line ever was “Hey.” …But what if he tested “Hey?” He’ll never know what could have been, but you can try A/B testing email questions within your own messaging.

Test #3: Change The First Line Of Your Email

The first sentence of your message often appears as a snippet of preview text in your recipient’s inbox. Getting this right could increase your open rates by up to 45%.

That’s roughly 40 to 90 characters to experiment with, depending on the mail app.

Test #4: Long vs Short Subject Lines

If your subject line is typically over 50 characters, this is a good opportunity for you to A/B test your subject line. It could be cut off in your prospect’s inbox. The iPhone only shows 35-38 characters in portrait mode (compared to 80 in landscape). And the Galaxy S4 constricts subject lines even further, showing only 33 characters in portrait mode.

Aim for less than five words.

Test #5: Capitalize All, One, Or None Of The Words In Your Subject Line

Interestingly, a Mailchimp study found that an entirely capitalized subject line resulted in slightly higher than average open rates.

Use a split test Campaign to find out what works best; send a title case subject line to Campaign A and a lowercase one to Campaign B.

Test #6: Personalize the Subject Line

Try A/B testing the same email body with two different subject lines—one generic, the other with a custom field that personalizes your message to the recipient. Some variants to test:

• Add and omit the word “you.” → How you can win in 2016 / How to win in 2016
• Include your recipient’s first name → Quick question, {!First Name} / Quick question (Here’s how to personalize up to 200 emails at once with custom fields)
• Mention your prospect’s role/occupation → Sales reps love this / People love this
• Use their company → {!My company} + {!Your company} = {benefit} / {!My company} = {benefit}

Pro tip: If you’re sending a Campaign, make sure you have a placeholder for any contacts whose information you might be missing (ex: you don’t have a first name for someone’s email address).

Test #7: Subject lines with high emotional value vs. low

Emotions drive actions. But did you know that you can actually measure the level of emotional impact your subject line has?

This calculation is known as Emotional Marketing Value (EMV). It’s used to assess how a group of words (like a subject line) might elicit an emotional response from a reader by appealing to one’s intellect, empathy, or spirituality.

A perfect EMV score is 100%, but a good score ranks anywhere from 60-80%. Email open rate A/B tests conducted by Coschedule found that subject lines with a higher EMV received higher open rates for the majority of their experiments.

Try scoring your own frequently used subject lines with this handy tool from the Advanced Marketing Institute and running a test of high EMV scores vs low.

Test #8: Put An Emoji In Your Subject Line

Try this strategy at your own risk :). While there is evidence that suggests using an emoji in your email subject line could increase open rates, ultimately its effectiveness boils down to your audience. Don’t do anything your recipient wouldn’t do.

A/B Testing Emails To Improve Your Reply Rate

Test each of the below to see what works for you. Optimize for replies by combining the highest performing variation of each.

Test #9: Longer vs Shorter Emails

Can your emails be scanned and digested in 8 seconds? If not, you might be losing the attention of your recipients.

Take an email template that you’ve sent in the past and cut it down. We recommend a free desktop app like the Hemingway Editor. Your old email serves as email version A, and the Hemingway haircut will be the B.

Test #10: Levels Of Personalization

It’s become the norm for sales and marketing emails to include custom fields like First Name, Company Name, and Job Title. If that’s your standard method of personalization, take it a step further and see what happens when you speak to someone’s interests, hobbies, or personal pain points.

Test #11: Email formulas

Chances are you’ve settled into an email template that you build off of as you go. But what if reorganizing your message could turn people who aren’t replying into responders?

There are basic scientific principles of persuasion and influence that copywriters have been using for years to reliably draw readers into their content. Here are 10 cold email examples you can test for yourself.

Save these formulas as templatesRe-use, track, make adjustments, boost repliesTry Yesware Free

Test #12: Send To More Than One Person

We recently examined over 500,000 sales emails sent by Yesware users and discovered that emails sent to one person while copying another had a reply rate 12% higher than emails sent directly to two people.

Could this tactic work for you? Try sending the exact same email to two groups — the A with two people in the To field, and the B with one in the To field and one on copy — to find out.

Test #13: Your Call To Action (CTA)

This is the part of your email where you ask your recipient to reply. The specific reason for them doing so could be just about anything: to schedule a demo, confirm they’re attending an event, etc.

CTA variations to A/B test:

• Putting your request towards the top of the email, vs at the very end
• Underlining, bolding or italicizing your ask, vs plain text
• Asking for a specific time to meet vs. asking “what works best” for them
• Asking for a shorter amount of time vs longer → ex. “Do you have 5 minutes?” vs “Do you have 15 minutes?”

To find your magic email, send a version with your usual question versus the optimized ask.

A/B Testing Emails for Click Rate

When a recipient clicks your email (tracking tools like Yesware flag this in real-time), you know that something appealed to them.

To capture their attention longer and understand what resources are resonating, try one of these A/B tests.

Test #14: Different Types Of Content Formats

Do your contacts prefer to watch, listen, or read the materials you send them?

Knowing which content formats get the most clicks will give you a better idea of the types of resources you should share with similar prospects in the future. Try linking out to:

• Written case studies vs video case studies
• Podcasts vs blog posts
• Webpages vs PDFs (you can use Yesware to track attachment opens.)

Test #15: Link Formatting

People click on links that are easy to spot. If you usually include your links in-line with your text, try a version “B” where you bold your link and enter it onto its own line. Like this:

See how the above link lists a metric? It’s another strategy to increase clicks. The more specific you are, the more persuasive your link is. Using a precise number makes your story more trustworthy in the eyes of your recipient.

Test #16: Link Text Length

According to MailChimp, 8-12 words is the link sweet spot. Here’s their analysis of 5.5 billion emails:

If your link text is usually well above or under 8-12 words, try out this tactic by A/B testing your usual email with a new, optimized version.

Test #17: Use Anchor Text To Appeal To Curiosity, Empathy, or FOMO (Fear of Missing Out)

Here’s how to test which emotional appeals work best to persuade your contacts to look at what you’re sharing:

Test #18: Vary The Number Of Links

Less is more. According to the ‘paradox of choice,’ theory, the more choices we have, the more unlikely we are to make any decision at all.

If you usually include two or more links, try narrowing it down to one.

Over to You

That’s all for now! What about you? Any A/B tips that we missed? Tweet us: @Yesware.