Why Focus Groups Fail

Title aside, this is more of a book recommendation for marketing people.  Read The Drunkard’s Walk, How Randomness Rules Our Lives by Leonard Mlodinow.   A terrific discussion on randomness, chance and probability, this book can help marketers understand how numbers can be used and misused.

I’m sure you’ve heard the adage about “lies, damned lies and statistics” but I still hear (and participate in, my bad) conversations where numbers and statistics are thrown around as if the universe WILL behave according to the percentages.  We all need to remember that even if a “majority” of any given group can be expected to behave in a certain fashion, there is still a chance that we could randomly fill a room and end up with all members of the minority who  behave in the opposite fashion.  (The dueling laws of large and small numbers, according to Mlodinow.)

If you are going to do any type of research, including focus groups, Mlodinow’s book is essential.  He cites focus groups at one point.  “Suppose you hold a focus group in which 6 people examine and comment on a new product you are developing.  Suppose that in actuality the product appeals to half the population.  How accurately will this preference be reflected in your focus group?”  According to Pascal’s triangle (one of the many mathematical theories Mlodinow explains and demonstrates), there are 20 ways in which the group members could split 50/50 (accurately reflecting “reality”) but there are 44 ways in which the group could split and NOT reflect reality.  Ouch.

There are other ways that focus groups fail.  Here are just a few:

  • Getting the wrong people in the room.  Who is the real buyer?  Who influences the purchase decision?
  • Asking the wrong questions.  Instead of asking, “Do you like this?” ask “Would you buy this?  Do you prefer this product over other similar products?  Would you purchase this product even if another product were half the price?  Would you continue to use this product if someone gave you another product?”
  • Asking leading questions.  People, especially kids, often want to please the questioner.  They want to give the questioner the answer the questioner wants.  Or, they don’t know what they really want and they “go along to get along” which leads to…
  • A strong personality among the group influences others. A leader emerges from the group and swings the group to his or her way of thinking.
  • The focus group doesn’t reflect the real world — I can say one thing in a focus group and do the opposite when I am in the “real world.”

Last year I saw a presentation from the CMO of a major consumer brand who claimed that they have ceased all survey and focus group activities and now rely only on measurement of physical reactions to their marketing.  By measuring brain waves, heart-rate and other body signs they can tell when a subject has an opinion or has made a decision, perhaps before the subject knows.

Do you still find focus groups useful?