Your subject line is the gatekeeper. The best email in your sequence generates zero results if the subject line fails to get it opened. Subject line testing is the highest-leverage email optimization available because a winning subject line improves the performance of every message you send from that point forward.
One good test, implemented consistently, can move your open rates by 10 to 20 percentage points. That difference compounds across every email you send for the next year.
In This Article
What to Test (and in What Order)
Test one variable at a time. If you change both length and tone simultaneously, you will not know which change produced the result. Start with the variables that have the highest potential impact and work down.
| Variable | Example A | Example B | Test this when… |
|---|---|---|---|
| Specificity | “Improve your close rate” | “How to go from 20% to 35% close rate in 60 days” | You want to know if your audience responds to concrete numbers |
| Tone | “Your proposal follow-up strategy” | “Are you following up on proposals the wrong way?” | You want to compare informational vs. challenge framing |
| Personalization | “A question about your pipeline” | “[First name], a question about your pipeline” | Your list has clean first name data |
| Length | “How to get more referrals” | “The three-sentence email that gets referrals from existing clients” | You want to know whether your audience skims or reads |
Specificity is usually the highest-impact variable for service business email lists. A subject line that contains a number, a timeframe, or a specific outcome consistently outperforms vague alternatives with the same audience. Test it first.
Personalization is overrated for cold lists and more valuable for warm ones. The first-name token in a subject line used to be novel. Most subscribers now recognize it as automation and discount it. Test it, but do not expect large results from name personalization alone.
How to Design a Valid Test
A poorly designed test produces confident-sounding but meaningless results. These are the constraints that make a test worth running.
- Minimum list size per variation: 200 recipients. Below this threshold, the results are not statistically meaningful. You could flip a coin and get similar data. If your list has fewer than 400 total subscribers, test concepts mentally rather than statistically and use the results from larger tests in your niche as directional guidance.
- Send both variations at the same time. Do not send variation A on Tuesday morning and variation B on Wednesday afternoon. Send rate is not the only time variable. Inboxes are different on different days and different times. Split your list randomly and send both at the same moment.
- Wait 48 hours before calling a winner. Most opens happen in the first 24 hours, but meaningful late opens happen in hour 25 through 48. Calling the winner too early can misread a slow-starting subject line as a loser.
- Measure the right outcome. Open rate tells you which subject line gets more opens. Also check click-through rate. A subject line that promises something the email does not deliver will get high opens and low clicks, which is worse than a more modest subject line that delivers what it promises.
Tools That Handle Testing Built-In
Most major email platforms support A/B testing. The implementation details vary, but the core functionality is the same: define two versions, set your split percentage, send, and review results after your waiting period.
- Mailchimp: A/B testing on subject lines, send times, and from names. The results view is clear and the winner can be sent automatically after a set time period.
- Kit (ConvertKit): Subject line testing on broadcasts with real-time results. Simple to set up. No option for automatic winner sending on the free tier.
- ActiveCampaign: Split testing with multiple variables and percentage-based distribution. The most flexible option. You can test more than two variations and set complex winner-selection logic.
- MailerLite: A/B testing built into the campaign builder. Clean results view. Supports automatic winner sending.
What to Do With the Results
Running a test without applying the results is a waste of the test. Apply winning insights immediately and systematically.
Keep a running test log in a simple document or spreadsheet: what you tested, what won, the margin of the win, the list size, and the date. After ten tests, look at the log as a whole. Patterns specific to your audience will start to appear. Maybe your list consistently responds better to specificity than curiosity. Maybe questions outperform statements. Maybe the day of the week matters more than the subject line itself.
These patterns are your audience telling you how they want to be communicated with. Apply the consistent winners as defaults in every new email you write. Your baseline open rate will drift upward as winning patterns accumulate. That drift is the compound return on your testing investment.
Also note what did not work. A subject line framing that consistently loses with your audience is as valuable as one that consistently wins, because it tells you what to stop using. Some freelancers and consultants discover that curiosity-gap subject lines (the kind that withhold something to generate clicks) backfire with their audience, who find them manipulative. Others find their audience loves them. The only way to know which is yours is to test.