Media Zone: It’s Just Testing9 Apr, 2010 By: Peter Aronow Response
During the 30 years that I’ve been in the DRTV business, one thing has stayed the same: everyone wants a quick, easy and inexpensive way to know what’s going to work or not. And despite the plethora of techniques developed over the years, only one — DRTV — effectively gives you the answer.
Back in the day, Time Inc. tested multiple creatives for their magazines by producing several animatics and testing them on air. These tests were run against a control (the current spot that was working) and the winner — when not the control — was produced and rolled out. Sometimes a new creative beat the control, sometimes not. But there were always meaningful differences in performance. Once the winning creative was produced, the results from the animatic tests always bore out in rollout.
Of course, many of us in the industry have tried or been involved with various forms of market research — focus groups, commercial eye tracking, copy testing, etc. But, as most of us have found, these testing methods — while psychologically comforting — only give you a very limited picture, while on air testing produces accurate, predictive results.
Going On-Air Is the Only Answer
If what you ultimately want to know is if someone will order a product from a TV commercial, then all the other testing, while interesting, is ultimately a waste of time. I realize various forms of testing make some clients feel better. It makes boardrooms happy and diffuses responsibility from marketing executives. Perhaps it even weeds out some real losers. Unfortunately, it also may kill some real winners. But it will never tell you what on-air testing tells you and what you need to know.
Testing on-air is the only way you really find out if someone sitting at home will write down the phone number or get up, pick up the phone, dial, stay on the phone talking to an operator (or listening to an IVR) long enough to supply credit card information and finally confirm an order (where they end up spending quite a lot more than they originally intended).
Ironically, testing is all the rage today in DRTV. From what I understand, it goes something like this: clients or consultants (not successful producers) with little or no experience or success developing creative write test scripts; these scripts then are presented to a predetermined test group via the Internet in written form.
This is shown to a potential audience of a half million people. If a product gets 50 orders, it’s considered a winner and a spot, usually not the creative that was tested, is produced. When the results are not good the product is dropped. But what was learned. Was the product or the offer unappealing? Was the creative at fault? Was the audience representative? Did the product fit into a category for which this testing had not performed well? And how can you test a visual medium like broadcast on the Internet?
Improving Your Testing
Over 30 years I’ve seen spectacular differences between creative for the same product and the same offer. Some were huge successes, others outrageous failures. And yes, it would be nice to know in advance which is which. But, at the end of the day, DRTV offers the best, most cost effective way to test products. All successful marketers know that. They know that if a production is close, that you can make it better, often successful, by tweaking the offer or the creative. Do testers know that?
While, in the past, animatics worked to predict what would be successful, today’s economics make DRTV the only real viable testing solution. In addition, you can simply roll out a successful DRTV spot before someone else copies it and beats you to market.
Now I know many people reading this will continue to support the idea of testing. I accept that. But I suggest you try to do it better. Find a successful writer/producer and have him or her write a script. Test several concepts against something you know works to validate the process.
Or simply accept DRTV for what it is — the best, most cost effective way to test a product or offer and work within the realities of this industry, as so many have successfully done before you. Not everything will be a success, but DRTV testing offers a very cost effective solution to bringing a product to market, and the economics of a success make it worth the investment.