A/B testing is a great technique to optimize the emails you’re sending to your customers to improve open rates, click-through-rates, engagements, and, ultimately, conversions. However, effectively preparing for the A/B test is almost as important as the A/B test itself. When planning A/B tests, you should follow these best practices before ever executing your test emails.

1. Identify your goals before A/B testing

Before you begin A/B testing, you must identify your goal. There’s no reason to run split tests (or complete any marketing function) without a goal in mind. What element are you going to test and why? What do you hope to accomplish with the results? How much does an email variation have to win by in order for you to consider the test results conclusive?

Asking yourself these questions and determining your goal before setting up your A/B test will allow you to execute your A/B test in the most effective way possible.

2. Only A/B test one element at a time

In order to get to the root of the efficacy of an A/B test, it’s important to only test one element at a time.

For example, if you are testing subject line in Email A, then you need to test it against subject line for Email B.

You don’t want to change the image and call-to-action in Email B to test against the subject line in Email A. You’ll be comparing apples to oranges, and you won’t get a clear result as to which subject line performed better.

Test the subject line against the subject line, body copy against body copy, call-to-action against call-to-action, and so on.

3. Understand the difference between elements and variations

While you only want to focus on one element during an A/B test, you can certainly run multiple variations of that test under the same conditions.

The term A/B testing suggests that you are testing two different emails against each other (with the same element, such as Subject Line testing), but it’s possible to test more than two variations in an A/B/C/D test, for example. Just keep it under control and don’t go overboard with the number of variations or you’ll be too far in the weeds.

Keep in mind that you are only testing ONE element in each variation, so if you’re testing the subject lines in four emails, you should write four different subject lines but keep the same body copy and send those emails at the same time of day on the same day of the week.

4. Make sure your A/B test sample sizes are adequate—and random

While running an A/B test with just 50 people is technically possible, it’s better to test with at least 1,000 people, so you have a large enough sample group to get true results.

The most common A/B testing type is A/B split testing, where you split your sample size down the middle and send 50% through Test A and 50% through Test B. It is possible to divide your sample size differently—for example, 10% get sent Test A and 90% get sent Test B, however, you’ll want to ensure your smallest test group has at least 500 people in it to yield the most telling evidence.

Also, randomly selecting the people in your test groups will help aid the efficacy of the test, but make sure the people in your test groups are from the same pool of people. For example, don’t send Test A to people from New York City and Test B to people from Philadelphia. If your pool is people from major East Coast cities, you should pull everyone from New York City, Philadelphia, Boston, Miami, etc. together, then randomly split that large pool of people into different test groups.

A/B testing shouldn’t be a one-time experiment. By following these best practices, you’ll be setting yourself up to execute, analyze, and optimize your A/B testing in the most reliable way to improve your email marketing.