The best way to approach “ranking” questions in your online research.
Using lists in surveys
When we’re conducting research for PR clients, many want to find data to support a “top 10” or “top 20” list — and who can blame them? A good list allows publications multiple avenues into the story, helping ensure it appeals to a wide variety of media outlets and has the best possible chance of coverage.
Just look at this example, from the West Virginia Department of Tourism. Ahead of World Tourism Day, OnePoll put together two lists — one of the “underrated” states that people want to vacation to, and another highlighting specific “underrated” vacation spots, after asking 2,000 Americans about their vacation destination habits.
This opened a variety of avenues for coverage: while the New York Post led on the idea of people vacationing in the same places over and over, the headline on TimeOut was, “The best underrated vacation destinations in the US.”
And, because this story highlighted specific locations, these lists also opened the door for more local and regional coverage. While these are often smaller outlets, they can be hugely valuable, and OnePoll’s story with the West Virginia Department of Tourism saw coverage on local Boston outlets like this one, which highlighted, “New Poll Says Massachusetts Is Underrated, Rhode Island Too.”
While it’s certainly true that lists are great for storytelling and popular with the media, we often find a disconnect between how clients want to capture that data, and how we recommend doing so.
So, what is a ranking question?
A ranking question asks respondents to compare and arrange multiple answer options in order. For example, in order of preference from “most preferred” to “least preferred,” or in order of importance, from “most important” to “least important.”
Is there a more effective alternative?
Yes, we think there is! Instead of giving survey respondents a list of choices and asking them to “rank” those options in order from top to bottom, we prefer asking respondents to “select all that apply” — for example, selecting any and all states they think are “underrated” when it comes to vacation destinations.
This is a better, more respondent-friendly question; if asked to rank the options, respondents are forced to make decisions about options they may not care for, or may not be knowledgeable about. If respondents have never traveled to West Virginia, Rhode Island or Massachusetts — and don’t know what those states are known for — they would still need to place one above the other when ranking the options. Allowing them to “select all” encourages them to pick the ones they have more information about, providing more accurate data within our results.
List-led media coverage
Asking respondents to select only the options they want therefore gives us stronger data — and it still allows us to pull a “top 10” or “top 20” list. As our Head of Research likes to say, “a ‘rank order’ is an output as opposed to an input.” It’s something we get from the data, once the survey is run.
This OnePoll study for Herbalife Nutrition is another example of great list-led coverage.
The 2022 story highlighted the entrepreneurs who’ve made the biggest positive impact on society, and landed on a variety of top-tier sites including the New York Post, Yahoo!, the Independent Español and iHeart Radio.
OnePoll has been conducting custom research and PR surveys for 20 years. We provide custom research services to brands, agencies and the media. As members of AAPOR — the American Association for Public Opinion Research, OnePoll researchers adhere to the principles and actions set out in the AAPOR Code. Our research team are also members of ESOMAR and MRS. Like to know more? Talk to our team or request a rate card.
Photo credit: featured image by Ellie Ellien
