How NOT To Use Focus Group Research
"To 18-34s - radio is 'free, but uninspiring.' That's one of the findings of a Jacobs Media study presented at today's Arbitron Consultant Fly-In..." -- Inside Radio, December 8, 2005.
I was at that presentation. The study consisted of six focus groups.
Imagine, an entire demographic group consisting of more than 68.5 million people, labeled as uninspired by radio by unscientific research conducted among what was probably fewer than 50 of its members.
That's the danger of focus groups. Without appropriate follow-up, statements by an unrepresentative few participants soon morph into sweeping generalizations that are acted on as fact. Good researchers know the most appropriate use of focus groups to help formulate questions for larger statistical studies. They never project focus group results to entire populations.
Public radio has done a pretty good job of avoiding this pitfall in the past. Focus groups findings have been tested by statistically valid phone studies, auditorium tests, and Arbitron recontact studies. That's why public radio has such a good track record of successfully applying research to programming decisions. And perhaps that's one of the reasons public radio had such a long streak of audience growth at the same time commercial radio listening was in a free-fall.
I was at that presentation. The study consisted of six focus groups.
Imagine, an entire demographic group consisting of more than 68.5 million people, labeled as uninspired by radio by unscientific research conducted among what was probably fewer than 50 of its members.
That's the danger of focus groups. Without appropriate follow-up, statements by an unrepresentative few participants soon morph into sweeping generalizations that are acted on as fact. Good researchers know the most appropriate use of focus groups to help formulate questions for larger statistical studies. They never project focus group results to entire populations.
Public radio has done a pretty good job of avoiding this pitfall in the past. Focus groups findings have been tested by statistically valid phone studies, auditorium tests, and Arbitron recontact studies. That's why public radio has such a good track record of successfully applying research to programming decisions. And perhaps that's one of the reasons public radio had such a long streak of audience growth at the same time commercial radio listening was in a free-fall.
posted by RadioSutton at 5:53 PM
2 Comments:
What might be an example of a good question for a focus group? Also, the fact that this study was "unscientific" seems a big problem, in addition to the "focus" of the focus group, no?
Focus groups are best used at the beginning of a research process to uncover the perceptions of a market segment -- whether for radio, commercial products and services, even political candidates. Used this way, focus group findings should be verified with scientific surveys of some sort.
Focus groups can also be used to test ideas. In this role, I think focus groups work best as a "disaster check" rather than to validate an idea. It's useful to know uf an appropriately recruited grop hates an idea. If they like it, validate it with quantitative research.
What happened with the Jacobs' focus groups is all too typical. Jacob's started the presentation by saying "all the usual caveats of focus groups apply" to these six focus groups. By the time the story reached Inside Radio readers, it was a Jacobs "study." No mention of focus groups let alone the number or nature of the groups.
Post a Comment
<< Home