Practical sampling decisions for your content marketing survey
Surveys are an essential part of many B2B content marketing programs. Unfortunately, response rates to surveys continue to decline across B2B sectors. As a result, the State of the Market program that once hit 600 completed surveys can struggle to reach half that number today. This post looks at factors driving the decline in response rates and offers steps you can take to help mitigate or slow the drop-off in your content marketing survey response rates.
What’s driving declining response rates
Due to budget constraints, content marketers typically rely on sending invitations to their existing customer and prospect lists as the primary source of completed surveys for their state of the industry and thought leadership programs. Historically this cost-effectively delivered robust results. However, over the past few years, two factors have combined to limit the productivity of this approach.
Too many messages: The underlying problem is the number of emails the typical white-collar worker receives – survey invitations get lost in the clutter of the audience’s inbox. Most research on workplace email indicates that the average person receives 100+ emails per day when you add up internal emails, direct emails with external parties, and marketing emails (including survey invitations). The number is even larger when you add in spam. On top of that, some individuals receive scores of messages from platforms such as Slack and Microsoft Teams. Amongst all these messages, individuals still need to get their daily work done. This leaves little time or attention left over for a survey invitation.
Survey and NPS fatigue: The intertwined rise of Net Promoter Score (NPS) and DIY survey platforms exponentially increased the number of survey invitations people receive. It can feel that almost every interaction you have with a software vendor or service provider in your personal or professional life results in an email invitation to an NPS survey. In that context, the invitation to participate in your state of the industry survey struggles to break through the clutter of survey invitations.
Implications & Recommendations
But all is not lost. Customer and prospect lists can still generate enough data to produce a credible state of the market and thought leadership content. But it requires a shift in thinking and some tactical actions. We present the following nine thoughts and considerations based on our two decades of conducting B2B surveys.
1. Focus on the Content, Not the Sample Size
A repeated pattern we’ve seen over the past two decades of helping clients is that content marketing teams, and the executives they report to, worry too much about sample size. While more data is always better, additional data points don’t necessarily add credibility once you reach a valid sample size. More importantly, the sample size has little, if any, impact on if prospects respond to digital marketing campaigns. If prospects are interested enough to download a report, they will read it regardless of how many data points the findings are based on.
In addition, regardless of how the vendor structured the sample, most readers of reports will assume that they need to take everything with a grain of salt. They understand that content marketing is marketing. What they are looking for is some interesting trends and vendor POVs. And prospects that get hung up on sample size will be skeptical of everything, and a larger sample size will not turn them into believers.
In most cases, the concerns about the sample size stem from internal perceptions, which can be slightly misguided, of what is “statistically” valid. And while B2B marketers should strive to collect as many surveys as they can, it’s also possible to waste a lot of time and effort trying to reach arbitrary sample size goals that have minimal impact on the success of the content marketing campaign.
2. Make it Interesting / Keep it Interesting
For a survey invitation to have a shot at being opened, the survey’s topic needs to catch the potential participant’s interest. They have to believe that the subject is relevant to them and that they will benefit directly or indirectly from participating in the research. Many people recognize that they can gain market insights by taking a survey and seeing what vendors are asking about. Others conceptually recognize the benefit of data that vendors release in their content marketing campaigns; and that for the content to be generated, they need to contribute. And some do it as a favor to the vendors they like.
One way to pique interest in a survey is to develop a compelling storyline rather than rely on general profiling surveys. See our post Be more systematic in your content marketing surveys for ways to identify topics likely to resonate with your target.
The work doesn’t stop there. You have to keep it interesting. Many content marketing surveys are designed to show market trends over time, so many marketers keep most of the questions the same year over year. Unfortunately, the reality of most B2B markets is that little changes in a year, or even in two years. As a result, the trendlines in the report end up flat, which isn’t that interesting for readers. In addition, hidden in this design is the practical recognition that customers and prospects need to come back every year and take the survey. It doesn’t take that many iterations of the same survey before these once-reliable participants drop out.
This is not to say that there shouldn’t be any tracking metrics in a survey. There should be. But to encourage engagement in a longitudinal content marketing program, the survey needs to explore new topics. These topics don’t need to be “hot” trends in terms of something specific happening in the market in real-time (e.g., the pandemic response, social justice movements, inflation). But it needs to at least provide a new point of view or angle on a trend that important to the market.
3. Financial Incentives
One of the most effective levers for increasing response rates is to provide a financial incentive for individuals to participate in the survey. The most common methods are online gift cards and donations to charities. Options include providing a gift card for each individual that completes the survey, structured as drawings for larger rewards, or as calls-to-action, such as only available to the first 100 respondents. While any financial incentive encourages participation, providing an incentive to every individual that completes the survey tends to work better than drawings or prompts to respond fast.
4. Set Broad Qualification Criteria
With today’s low response rates, it is imperative that you do not screen out too many people who respond to your invitation. For example, your ideal design may only include respondents with director-level and higher titles that work at companies with 1,000+ employees. However, these criteria place a cap on the number of individuals on your lists that will fit that profile, and while we noted earlier that you shouldn’t get too hung up on sample size, you can have credibility problems if it is too low. There are likely questions in your survey that a manager-level respondent can provide answers to or apply to a company with 700 employees. And there may be interesting points of comparison. In addition, in your analysis and reporting, you can filter and sort respondents on the backend as needed.
It is important to note that this approach only applies to invitations you send to your customer and prospect lists. If you use an online panel, you should keep your criteria focused on the profile of individuals of most interest – you will pay for each respondent.
5. Digital Promotion
Sample size is a numbers game. You need to get the invitation in front of as many people as possible. Teams that take a digital marketing campaign approach to their invitations tend to collect more surveys than those that rely solely on their customer and prospect lists.
6. Keep it Short
If a B2B respondent is willing to start your survey, they will give you more than just a couple of minutes of their time. Therefore, your survey doesn’t need to be the 5-10 question surveys common in consumer research. Still, you want to be respectful of the participant’s time – you are getting more from the exchange than they are. We recommend thinking about your survey in terms of clicks instead of question numbers and then having a maximum of 50 clicks. A click is whenever a participant has to read a question and select an answer. For example, picking an answer from a list of possible responses to a question that asks, “Which of the following best describes…,” represents a single click. Ranking the importance of 10 attributes equals 10 clicks – the participant needs to read each attribute, decide on their answer, and click the appropriate box. Thinking in terms of clicks rather than question numbers will help you keep a more accurate count of the length of your survey.
Content surveys can get long because different internal stakeholders such as product marketing, product management, and sales want to add questions to the study. It is important to remind these stakeholders that the purpose of content marketing surveys and their outputs is demand generation or thought leadership, not to inform internal strategies. We recognize this is easier said than done but still worth keeping the principle in mind.
7. Benchmarking Data / Participant Only Data
Although content marketing teams often overestimate the value of benchmarking data, some participants are interested in how their responses compare to others. Providing this information can motivate some individuals to take part in the survey. Another tactic many content marketers use is to promise access to a subset of data that will only be available to individuals that participate in the survey. The value of this incentive relies on the extent to which potential participants are interested in the data.
8. Early Insights.
Content marketing survey invitations almost always include a line that says the respondents will be the first to see the results. Unfortunately, this is not motivating for most potential respondents. They know that the vendor sponsoring the research will push out the content as far, wide, and fast as possible. They also know that a sales rep would be more than happy to walk them through the findings
9. Online Panels
If the practical realities of your response rates or lists size limit the feasibility of collecting a meaningful number of surveys, you can turn to online research panels and expert networks for additional surveys. These sources are often used for other, internally focused research projects such as product development, market strategy, and communications. Using these sources requires two types of investment. The first is budget. Depending on the audience you want to reach, the cost per survey can range from $50-125 per respondent, which quickly adds up. The second is effort. As good as some of the panels are in screening out fraudulent respondents, on average, you will likely need to reject 30-50% of the responses you receive. This requires manually reviewing the responses of each respondent.
No Silver Bullet
None of the recommendations above represent a silver bullet solution to declining response rates. Instead, the best approach is to use a combination of levers you have available and be realistic internally about what you can achieve.
If you’d like to learn more about how Isurus help clients design and collect content marketing data, you can reach out to us here.
The following posts provide additional thoughts on marketing and communications.