​The Danger of Customer Research and Big Data – "The Emperor Has No Clothes"

So Donald Trump defied every major political poll.  The NY Times showed Hillary Clinton with a 92 percent lead for many weeks prior to the election. All those fancy polls and big data were not even worth the paper that they were printed on.  The pollsters didn’t reach the right or enough people, people lied, or some combination occurred.

This same phenomena happens in product development. Products fail big time despite doing all the market research, big data analysis, and prototype testing.  Why?  Customers sometimes lie.  People don’t really intend to but they will have a tendency to tell you what you want to hear.  We learned about that tendency in the children’s story “The Emperor Has No Clothes”.
​
Social desirability bias describes the tendency of respondents to answer questions in a way that they believe to be socially acceptable.  In other words, respondents might give answers to be polite, not to offend, or to make you happy.  How did you respond to your girlfriend asking, ”Does this dress make me look fat?”  How did your EX-girlfriend react when you told her the truth?  (By the way, the correct answers are either “No way!” or “No Hablo Ingles.”)

Here’s a possible, classic scenario: a company invites “friendly” customers to provide feedback on a prototype product.  The customers are greeted by product marketing and engineering managers involved with the product development who then happily share how excited they are (“We spent eons innovating!”).  The first mistake is going for all “friendly” respondents.  The second mistake is sharing too much about the product and the personnel.  Put yourself in the shoes of the respondent:  “I’m drinking your free coffee and eating your free donut.  You just told me how great your product is supposed to be.  I don’t want to hurt your feelings.  There’s a chance that you won’t invite me back if I am too negative.”

People are also biased and selective in deciding what they want to hear.  We filter out naysayers and label them as outliers in the data. The pollsters in the election either did not pick up the anger in particularly the rural masses or underestimated it.  One likely factor is that the pollsters, being urban and college educated, simply filtered this out.  Or they thought that it wasn’t important enough to buck their fellow pollsters.  

The point is that customer research, customer testing, and data analysis are important but take it with a grain of salt.  Gather all the data from all sources.  Talk to your field sales and service staff to get the quantitative and qualitative data.  They are your boots on the ground who know the customer best, can see trends firsthand, and most likely to be honest with you.  And use your gut instinct and experience.