Once during my younger years, I was presenting test results to a client along with recommendations on what we should mail next.

The client acknowledged that the test results supported my recommendation but said, “Our president doesn’t like that particular copy.”

My immediate response was “So?”

Admittedly, it wasn’t my most tactful hour. But when it comes to successful direct mail, it really doesn’t matter what we think is the best offer, copy or package design.

What matters is what our customers and/or donors respond to.

Fortunately, direct mail gives us a reliable method for measuring what motivates our customers to respond. And when we know how new elements of a mailing—lists, offer, copy, format and timing—affect response, we can drive our direct mail campaign toward greater profitability.

What we can’t do is ignore valid test results because they weren’t what we expected. Test results are the voice of our customers and they will lead us to success if we listen.

When to test

 A bad economy isn’t a good reason not to test. In many ways, it makes testing more imperative than ever. Not only will testing make your future mailings more cost effective, it will position you to capitalize on an improving economy.

The best time to test is when you have a successful control.

Too often, clients don’t want to invest in new tests because “our control is still working.” But waiting for the control to fatigue before you begin testing can mean disaster for your cash flow. Test from strength. However successful your control is, and however long it’s been the control, it can be improved.

What to test

Typically, the more dramatic a change you make in a package, the more dramatic the difference in results.

For example, when you need a breakthrough, test the components that have the greatest influence on the mailing’s success—lists, offer, format and copy. Forget about testing minor changes on page 3 or the color of the return envelope. Test big things for big results.

However, for clients mailing larger volumes, “tweaking” the control for incremental gains often makes sense.

For example, Client A and Client B both have an average response rate of 1%, with a $25 average transaction. Both test a new package that lifts results by 10%. The only difference is that Client A has an annual mailing volume of 1 million pieces while Client B mails 10 million pieces every year.

For Client A, with its smaller mailing volume, this 10% lift means an annual difference of $25,000 in income—hardly enough to cover the cost of the test. But for client B, the same 10% lift means $250,000 in additional income and that’s enough to have an impact on the organization.

But whatever you decide to test, do the math first. Testing a package that can’t recover its test cost in a reasonable time, or that requires an unrealistic rate of response to beat the control, doesn’t make sense. (See How to Use a Pre-Event Routine for More Profitable Direct Mail Testing.)

How much to test

When deciding how many pieces to test, the important thing isn’t how many test packages you mail but how many responses you get back.

Your test volume should be no fewer than the number you need to produce statistically reliable results. Personally, I like to see 100 responses. But mathematicians, using formulas I won’t even pretend to understand, say as few as 39 responses can be statistically valid with a 90% to 95% probability of accuracy.

You can use an online calculator to tell you how much to mail, but the concept is easy. Just take the control’s response rate and see how many pieces you need to mail in order to produce 100 responses. For example, with a 1% response rate, you would need to mail 10,000 pieces to produce 100 responses.

As a rule of thumb, you’ll want to limit your total test volume to no more than 20% to 25% of the total mailing. This way, if the tests fail, the returns from your control will help absorb the loss and preserve the operating budget.

How to Test smart

You will always have someone saying, “We’ve never done this before.” Or “I wouldn’t respond to this.” Or “This isn’t creative enough.”

But you can remind the naysayers that the purpose of testing isn’t to be creative. It’s to make money. And of course we haven’t done this before. That’s why we’re testing it now. And whether the office skeptics would respond to the mailing really doesn’t matter. You’re not mailing to them.

Testing is essential to profitable direct mail, but test smart.

• Don’t take shortcuts. Do the math first.
• Test things that matter most—lists, offer, copy and format.
• Don’t be afraid to test things that increase your package cost. Typically, you’ll increase profits more by adding features and benefits than by eliminating components and benefits.
• Don’t get “test happy” and schedule tests just to satisfy someone’s curiosity.
• Before you mail, have procedures in place to credit the mailing with responses to the 800 number or website.
• Evaluate test results using rollout costs. Test costs do not accurately reflect how future mailings will perform.

Taking time to understand and practice the fundamentals of direct mail testing goes a long way toward ensuring your campaign’s success. And whether you’re a startup or a “mature” mailer, the three most important rules of direct mail are the same: Test. Test. And Test.

Test smart and have the discipline to act on the results.

Simply put, your direct mail offer is the “deal” you promise the recipient. It’s what you promise the reader and what you ask in return.

Your offer needs to be specific and to clearly state how it benefits the prospect. It includes the product—or for a fundraiser, the organization’s mission or project—the price or asking amount, terms, incentives, guarantee, etc.

And, of all the components of your mailing—other than the list—the offer is the most important element of your success. If you’re looking for breakthrough results, here are 17 quick ideas to consider:

1. Offer something free—a free gift for responding, a free trial period to test the product, free shipping, etc.
2. Offer something new. Along with “free,” “new” is still one of the most powerful words you can use.
3. “Flip-flop” your offer. Rather than emphasizing “You’re paying $400 too much for auto insurance,” test “You can save $400 on your auto insurance.” You can sell a product at “50% off” or offer the reader “Buy one and get one free.” You can offer a product for $24.99, or test selling it for “$29.99 with free shipping.” In each of these examples, the cost to the mailer is the same, but repositioning the offer can lead to a big difference in results.
4. Use specifics. Rather than offering to save the reader $400, offer to save him or her $398.43. Although it’s less of a savings, the specific dollar amount can attract more attention and give the offer added credibility.
5. For political fundraisers, rather than ask for money to “help elect” a candidate, raise money to “defeat” the opponent.
6. Use goals that people can relate to. For example, rather than raising money to feed millions of hungry people, ask for help to feed one hungry child. Rather than cutting taxes by billions of dollars, offer to save each taxpayer $1,023.
7. Where possible, don’t use percentages. What does it really mean to save 25%? Instead, offer to “Save $19.” Rather than “Sixty percent of all consumers agree,” use “Three of every five consumers agree.”
8. Test a premium. If you’re already offering a premium, test a new one. (See Why Test Premiums)
9. Test new prices and/or asking amounts.
10. Provide a guarantee. Everyone can offer a guarantee; if you don’t think you can, you shouldn’t be selling the product. Even a nonprofit can offer a guarantee to use the donor’s money wisely or promise to refund the donor’s initial contribution if they ever become dissatisfied with the organization.
11. Include a trial offer. Like the guarantee, a trial offer limits the prospect’s risk and reduces their concerns about responding.
12. Make the offer “For a Limited Time Only.” By including a specific deadline in your offer, you can force the prospect into making a decision.
13. Offer “Early Bird” savings or gifts. (“If you act within the next 10 days, you get…”)
14. Test “Yes,” “No,” or “Maybe” options with peel-off stickers that the recipient transfers to the response form, or include the options with simple check-off boxes. Giving the prospect Yes or No options typically increases response. And adding a Maybe option often boosts response even more.
15. Offering a “Charter Membership” gives the prospect added recognition and is a simple way to boost response for a new organization. Also, consider offering an “Introductory Price” as a cost-effective way to acquire new customers.
16. Offer a “Bill Me Later” option or installment payments. The “Bill Me Later” option is common for subscriptions, and offering installment payments can make large-ticket items seem more affordable.
17. Regardless of your offer, sell only one thing at a time. This is one of the oldest rules of sales—ignoring it can prove costly.

You often get the best results by combining two or more unique offers, so don’t be afraid to mix and match from the above list. If you find a premium that works, perhaps adding free shipping or a discount will increase the mailing’s net revenue even more.

And this brings up an important point. The measurement of success is net dollars. If you need to increase your mailing’s cost to sweeten the offer, don’t let the higher cost be the one thing that stops you. If you believe the new offer has a reasonable chance of increasing the mailing’s net return, it’s worth testing. Focus on net dollars. (See “The What, When and How of Direct Mail Testing.”)

When you need breakthrough results, test your offer. Other than careful list selection, nothing will make a bigger difference in your success.

We all know the importance of direct mail testing, but too often money is wasted on unproductive and unwise tests. Here are 21 basic rules worth reviewing to get the most from your marketing dollars:

1. Before you start production on any test, do the math first. How long will it take to recover your test costs, and what increase in results will you need to beat the control? Proceed with the test only after showing that there’s a reasonable chance that it can economically increase response.

2. Don’t test just because you’re curious to know “what if.” Have a solid plan of how you’ll turn the test results into a profit before you invest in the test.

3. Be sure you test a sufficient quantity to obtain reliable test results.

4. “Replicate” each test when possible—rather than mail one test cell of 10,000 names, split the names into two equal groups and mail the same test to two groups of 5,000.

5. When possible, mail the same number of pieces for each test.

6. Create two test-size cells from the names that will receive your control mailing and use them as the baseline for measuring your tests’ success. These “control cells” will eliminate any bias that mailing to a larger body of names might cause, and, if there’s a production glitch, they can protect your cash flow. Now, rather than hold the entire mailing until the production problem is corrected, you can mail most of your control and hold these test-size cells until your tests are ready to mail.

7. Constantly test for new lists. Nothing will make a greater difference in results. And when you get a new package breakthrough, review the lists you tested earlier and see if your new, better-performing control makes them worth retesting.

8. Rely on the experts for list selections and data work, but make certain you understand what they’re doing. You don’t necessarily need to understand the math behind modeling, but you do need to be certain that the experts understand your objectives.

9. Test new offers. Second only to lists, you’ll get the biggest return from your test dollars by testing new offers.

10. Test new graphics for the outer envelope. These are among your most cost-efficient tests, and they can help keep a control “fresh.”

11. Mail your tests and the control at the same time and, as much as is feasible, from the same postal outlet.

12. You can’t argue with the test results, but don’t follow them blindly. Pyramid—roll out with ten times the number of pieces tested—and reevaluate and/or back test.

13. Don’t get test-happy. Test only those things that can make a meaningful difference to your mailing’s success. It may make sense to tweak the control for incremental gains if you’re mailing in large volume, but if not, limit your tests to the big things—lists, offer, format and copy.

14. Follow up quickly. And minimize your risks and improve the reliability of your results by using proven lists for your tests.

15. More often than not, you’ll increase profitability by increasing the package’s cost—not by taking away from it.

16. If you’re not using a premium, test one. If your control uses a premium, test a new one.

17. If your control includes a brochure, test without one. A brochure often distracts readers and delays their move to the response form. (This is one exception to rule 15.)

18. Make sure your tests receive full credit. Are replies to the 800 number and the website being counted?

19. When calculating how well your test will perform against the control, use rollout costs.

20. Measure your results by net dollars—not response rate or cost per order. If you can’t spend it, it doesn’t count.

21. And, as elementary as this may seem, be certain that you have a competent system in place to receive, count and report results.

You don’t need to be a rocket scientist to succeed in direct mail. But you do need to know what, when and how to test. And you must have the discipline to follow the results.

Let these 21 rules guide you and you’ll have a great start toward a more profitable direct mail campaign.

What to test

In his book, Secrets of Successful Direct Mail, Dick Benson states, “Any idea you honestly believe can economically increase response is worth testing.”

The key words are “economically increase response.” But what is economical? Typically, the more dramatic a change you make in a package, the more dramatic the difference in results.

For example, when you need a breakthrough, test the components that have the greatest influence on the mailing’s success—lists, offer, format and copy. Forget about testing minor changes on page three or the color of the return envelope. Test big things for big results.

However, for clients mailing larger volumes, “tweaking” the control for incremental gains often makes sense.

For example, Client A and Client B both have an average response rate of 1%, with a $25 average transaction. Both test a new package that lifts results by 10%. The only difference is that Client A has an annual mailing volume of 1 million pieces. Client B mails 10 million pieces every year.

For Client A, with its smaller mailing volume, this 10% lift means an annual difference of $25,000 in income. But for client B, the same 10% lift means $250,000 in additional income.

While the smaller client might appreciate an extra $25,000 to spend, it is probably not going to have a major impact on the organization—especially after accounting for test costs. On the other hand, a quarter of a million dollars is “adult money.”

But whatever you decide to test, do the math first. Even if the test beats your control, it won’t make sense if it takes too long to recover your test costs or requires an unobtainable rate of response.

When to test

The best time to test is when you have a successful control. A too-common mistake is to delay testing new concepts because “our control is still working.” By this reasoning, you wouldn’t test new concepts until the control takes a dive, and this is a disaster for cash flow. You may even find that it’s time to update your résumé, because there’s not going to be enough cash flow to make payroll.

Instead, test from strength, not weakness. Virtually every mailing is an opportunity to test. However successful your control is, and however long it’s been the control, it can be improved.

How much to test

When deciding how many pieces to test, the important thing isn’t how many pieces you mail with your test package. What’s important is how many responses you get back.

Your test volume should be no fewer than the number you need in order to produce statistically reliable results. Mathematicians using formulas I won’t even pretend to understand tell me that as few as 39 responses can be statistically valid with a 90%–95% probability of accuracy. Personally, I’m more comfortable with closer to 100 responses.

You can use an online calculator to tell you how much to mail, but the concept is easy. Just take the control’s response rate and see how much you need to mail in order to produce 100 responses. For example, with a 1% response rate, you would need to mail 10,000 pieces to produce 100 responses.

As a rule of thumb, you’ll want to limit your total test volume to no more than 20%–25% of the total mailing. This way, if the test fails, the returns from your control will help absorb the loss and preserve the operating budget.

Smart testing

Testing is essential to profitable direct mail, but we need to test smart. Evaluate the success of the test based on its rollout cost, but always do the math first and make sure you can pay back your full test costs in a reasonable time.

And remember Dick Benson’s advice and test only what you honestly believe can economically increase response.

J. Paul Getty once said the three keys to wealth and success are, “Rise early. Work hard. And strike oil.”

That may be good advice. But for those of us who must depend upon something other than striking oil, I say the three keys to success for a direct marketer are to TEST, TEST, and TEST!

But before we test, we must recognize that not all tests are productive or cost-effective. My next post will be “What, How and When to Test” but in the meantime, here are 7 costly mistakes that you’ll want to avoid when testing:

• When you’re looking for breakthrough results, it makes no sense to test small things—the color of the return envelope, minor copy changes buried in the letter or a different paper stock, for example. When you need a breakthrough, test the big things—list, offer, format and copy—to get big results.
• Do the math first. Before you make the test mailing, you need to know how much of a “lift” you need to break even. Make sure you have a reasonable chance of winning. On the other hand, don’t conclude that the test “costs too much” without doing the math. Increasing your package cost by 50% doesn’t necessarily mean you have to increase response by 50%. Much will depend upon the size of your average order, so do the math first.
• Don’t ignore past test results. At times, we get valid test results, but they weren’t what we expected so we’re tempted to ignore them. Your test results are the voice of your customers, so listen to what they’re saying—even if it’s not what you expected to hear.
• Don’t test more than one thing at a time—or test everything. You can’t test a new format to a new list and conclude that the format made the difference. However, you can test new copy and a new format—to the same list—as long as you recognize that you’re testing the package, not the format or the copy.
• Don’t think that just because something worked for another mailer, it will work for you.
• Rather than become stuck in a “make it cheaper” mode, test adding features and benefits to your package. More often than not, you’ll increase profits by adding to the package rather than by taking away features and benefits.
• How many times have you heard, “I don’t need to test; my control is still working”? The best time to test is when your control is working. Don’t fall into the trap of complacency. Otherwise, you’re going to put a serious hurt on your cash flow.

To succeed in direct mail, you don’t need to be a rocket scientist, but you do need to understand the fundamentals of testing and have the discipline to follow the results.

Test results are like a road map, and when you follow them, they’ll lead you to success.