Once a month? Twice? Three, four, five times? How much is too much, and how much is too little? Those of us who work in market research know that this is a question often posed by companies that recruit panelists to participate in their surveys and other research projects. All of us want to determine the optimal frequency for surveying.
In other blog posts, we shared the results of our internal investigation, in which we conducted mystery shopping of online panels. Our team registered for ten online panels, including the Netquest panel, in order to compare different characteristics of panelists’ experience.
This time, we’re trying to determine how frequently surveys should be sent to panel members.
Is There a Perfect Frequency?
Nobody likes to get dozens (or hundred) of promotions, discount alerts, and newsletters in their inbox every day. We can include survey invitations in this category, too.
The frequency with which you send these invitations is critical if you don’t want people to think of your emails as spam. You don’t want your emails to get lost in recipients’ inboxes or, worse, automatically flagged and sent to the spam folder.
When it comes to online surveys, frequency means the time that passes between two invitations to participate. In other words, how often these invitations are sent.
This is a topic that gives us a lot to talk about, because there is no single standard or consensus about the ideal number of surveys to send. No conclusive research study has been carried out on the optimal frequency for surveying.
Implications for Panelists
The frequency with which surveys are sent has a directly influence on the professionalization or non-professionalization of panelists, who end up responding whenever they can in order to receive a greater number of incentives.
Four of the ten online panel companies analyzed send surveys every day. From panelists’ point of view, this is great, because they receive better compensation (the more surveys they complete, the greater the incentive they receive).
In that vein, some argue that there are advantages to sending surveys more frequently, since panelists become better at interpreting questions, scales, and so on as they complete more surveys.
That said, other researchers worry that this practice compromises the quality of the data collected. Thus, even though there isn’t one ideal frequency, many are of the belief that “less is more.”
One argument that favors this line of thought is that, in addition to helping prevent panelist professionalization, it encourages panelists to pay attention, because, at the end of the day, they’re receiving fewer invitations to participate.
Additionally, at Netquest, we monitor surveys to ensure quality. There are ways to make sure panelists are completing surveys in good faith: we can use trick questions, monitor how quickly they complete the survey, check responses for consistency, and so on. This infographic includes a checklist for ensuring quality data.
The Dilemma: More Invitations or Fewer?
The way we see it, it’s not a good idea to inundate your panelists with emails. It gets annoying, not to mention it’ll eventually impact your brand and participant interest.
Regardless of the “ideal” standard, at Netquest, we limit the number of survey invitations that each panelist receives, as well as the total number of registered participations per person. We opt for a fairly low survey frequency, compared to other online panels.
In our internal investigation, we found that four companies invite their panelists to complete a survey with a frequency of once a month, or even less often. At Netquest, we recommend maintaining a minimum of one invitation per month so that panelists don’t forget about the panel.
The Netquest panel sends between one and three invitations every month. We have found that the response rate in our studies improves when invitations are sent once every 8-14 days.
As is clear from the graphic above, 40% of panels send daily invitations. Regardless of the standard applied, these panels must share this information with the researchers who hired them.
Why? This is a very important point that all professionals should know: the people participating in their studies are also completing several surveys a day. This is a key fact to keep in mind when safeguarding and monitoring the quality of the data collected.
If you liked this post, stick around. Take a look at our ebook and read more essentials of online data collection.
