The Difference a Test Makes

5 steps for creating simple tests that improve email marketing results

by Gail Goodman, Constant Contact CEO

What if you could make a slight change to one of your emails and get
a 20 percent lift in opens or a 10 percent increase in sales? You would
do it in an instant. But how do you know what element to change? That’s
where testing comes in. Email marketing makes it easy to quickly test
important elements of your email—at very little or no extra cost. With
testing, you can find out what factors influence the success of your
email. Follow these five steps to create an effective, measurable test.

Step One: Decide What to Test

Because testing with email is so easy, it’s often tempting to test
many elements all at once. You should start by testing just one. Why?
If you test more than one element in the same email, it is challenging
(and sometimes impossible) to determine what exactly influenced the
response. Here are some easy and telling tests to start with:

  • Subject lines – Create two different subject lines for the same email communication. For example, a boutique
    owner just added a home and garden section, and she wants to get the
    word out to her customers. Here are the subject lines she’ll test.

    • Subject line #1: New! Home and garden section added
    • Subject line #2: Get what you need for your home and garden

  • Long versus short copy – Is less really more? Create a shorter version of your current
    newsletter with teasers and links to your website. Or create two
    versions of a promotional email. Keep one very short and to the point
    and make the other a little longer by adding additional, useful

  • Special offers
    Create two different offers. For example, an online bookseller wants to
    get rid of last season’s bestsellers. He sends the following offers to
    see which one gets a better response.
    • Offer #1: Buy 3 books and get 1 free
    • Offer #2: Buy 3 books and get free shipping

    Other tests could include the time of day or day of the week you send, with
    an image or without, types of calls-to-action, and the placement of a
    call-to-action button or link. I’m sure that you will come up with
    other areas you would like to test as well.

Step Two: Decide How to Measure Success

What will you measure to determine success? Possibilities include
website traffic, response to an offer, sales, opens, and
click-throughs. Whichever you decide on, be confident that you can
attribute an increase (or decrease) in the area you measure directly to
the email you send. The easiest place to start is with your email
campaign opens and click-throughs, data that your email marketing
service provider makes available to you.

Step Three: Determine How to Divide Your Email List

When it comes to who you will send your test to, you have two
options. You can either split your entire list in half and test one
against the other or take a random sample and do a pre-test.

A pre-test is an excellent way to find out what works before sending
the email to your entire list. This knowledge can greatly improve your
overall response rate. It also protects you from sending a poor
performing email test to a large portion of your list—and wasting your
efforts. To pre-test, choose a random sampling of 100 people from your
master list, then split that in half and send each half one of the two
test campaigns.

Step Four: Test, Measure, and Declare a Winner

Once you have everything ready, send your test emails. The great
thing about email is that you get your results quickly—within a 24 to
48 hour period you will know which email communication got a better
result (it takes weeks when testing with direct mail). Declare your
winner, send that email to the remaining members of your list, and
watch the results come in.

Step Five: Have Fun and Keep it Up

Did I mention that testing is fun? Make a guess of which version
will win before you send, and see if you are right. What’s amazing
about testing (and what proves its incredible value) is that many times
the results are not at all what you expect.

Let your customers, clients, or members tell you, through their
actions, what they respond to best. This method is an excellent and
trustworthy way to improve your emails. Test often. You might be
surprised every time!


1. Export the entire list you want to send to (only export the data you
need for your test—most likely first name and email address) from your account, to an Excel file.

2. To get a random sampling of your list, go to the first empty cell in the first row and
type =RAND() in. Grab the corner of that cell with your mouse and pull
down to the very end of your list. Each person on the list will be
assigned a random number.

3. “Select All” (“Control” + “A”), right click, select “Copy”, right click, select “Paste Special”, then select “Values”

4. Go to “Data” and select “Sort.” Sort by the column with the random numbers.

5. Save the first 100 contacts (your random sampling) and delete the rest.

6. Divide this list into two (1-50, 51-100) and save one list as A (in one
Excel file) and the other list as B (in a second Excel file).

7. Import both of these files into your Constant Contact account. Name
them for the campaign you are running. For example, “Father’s Day
Promotion A” and “Father’s Day Promotion B.”

8. Create the two campaigns you want to test.

9. Select list A for one campaign and List B for the other.

10. Send and watch the results come in.