If you are going to fail, fail early. Those words were from Prof. Duncan Simester. It’s amazing that many organizations tend to operate like battleships – difficult to change course quickly once they get started (if at all). We tend to have an “all or nothing” approach and operate in a serial manner in pricing, product launches and marketing campaigns. The problem is that the initial hypothesis may be completely incorrect and we will not know that we have a major failure until the very end. What used to work in the past is no complete guarantee that it will work in the present or future. Customers and markets change. A new variable may have been overlooked.
That’s where testing and experimentation comes in. Try different pricing schemes, marketing messages, sales scripts to test customer acceptance and resulting sales. Without experimentation, we will see signs of problems but will ignore them, or even worse, throw more money and resources thinking that will resolve the core issues. It will then reach a point when many encounter the “too big, too late to let it fail now” dilemma...except that it will fail. Consider the recent consumer backlash against Verizon Wireless' $2 “convenience fee” for credit card payments, Bank of America’s debit card fee, and Netflix’s rate hikes. Why didn’t they test this in a small segment instead of going for a nationwide rollout? As a result, they faced an avalanche of ill-will, criticism, and subsequently had to reverse course with at least Netflix losing business.
Testing and experimentation is part of life. You did it when you were dating. You try different things to see what works or fails. (Did you get slapped when you used that line? Okay – you may not want to try it again.) In war, armies do probing attacks to test the enemy’s readiness and reaction. The “Charge of the Light Brigade” approach can mean tremendous loss if it fails. McDonald’s offered Fish McBites when I travelled in Texas and Chicago in March but I don’t see them in the Northeast US – they are great at testing and experimenting with their products and promotions.
With the internet and social media, an organization has no excuse not to do business experiments. Test pricing and marketing strategies in different areas or customer segments,
Some Basic Steps to Experimentation:
• Design the experiment
Start with a hypothesis. Measure what happens in a test group versus a control group. Be clear on what you need to measure to produce a decisive result—and whether that’s a metric you can track.
• Keep it Simple and Think Short Term
Small and simple experiments are easier to manage, less costly, and a short timeframe won’t test the patience of upper management. There is also a likelihood that word will spread because many customers are part of worldwide organizations.
• Slice and Dice the Data
Remember that there will be lots of data and that statistics can mislead. Segment and dissect the data. For example, you may want to segment by gender, age, geography, SIC code, etc. Most actions affect some customers more than others. Look for subgroups within your control and test groups.
• Act on Facts
Nothing but a success in a testing environment should be rolled out more broadly. But neither should failures simply be scrapped. Refine the hypothesis on the basis of the results and consider testing a variation. Capture what’s been learned so resources aren’t wasted proving the same thing again!
Two experts and proponents of experimentation are Thomas H. Davenport and Duncan Simester (click on names for links to great step-by-step guides).