Meta surveys vs. detailed explorations: When to use each

This organizational chart of trust has made its way around LinkedIn and Twitter. It’s a great reminder that many elements contribute to feelings of trust. We can use it to ask ourselves how well we and our organizations perform in these areas. It also provides a framework for thinking about other measures such as quality.

But should organizations use frameworks like this to formally measure attributes such as trust, quality, loyalty, etc? This question highlights two ends of a continuum in how market research is conducted today.

Historically market researchers have devoted their energies to pulling apart the dimensions that drive macro perceptions and behaviors such as trust or loyalty. Qualitative laddering techniques and quantitative regression modeling are common tools used to drill down to the core drivers. Two primary principles underlie this approach. The first is that while many factors influence the macro perception—trust in the case above—some factors are more important than others. Companies can identify the tactics that will be most effective in moving the macro needle. The second principle is that people often do not know or cannot effectively articulate their perceptions and respond with either automatic or overly cognitive answers.

This approach has provided, and continues to provide, valuable and actionable insights in consumer and business markets. However, some researchers believe that level of insights provided does not offset the cost, time, and complexity associated with the approach. They contend that long surveys are tedious are reduce cooperation rates. They also believe that the results can be too nuanced and “researchy” for the product and marketing teams that have to use the information. Researchers in this camp believe that most dimensions are highly correlated so asking a few macro questions is enough. They also point out that shorter surveys are easier and less costly to field and the results provide a more straight forward story for those that need to take action on the research. The DIY survey platforms are based somewhat on this philosophy. It is also the general principle behind the Net Promoter Score.

Both approaches have their place.

Using the trust example above using a short survey—1. How much do you trust company A? 2. What would make them more trustworthy?—would identify where problems exist, the magnitude of those problems and the general causes.

Unfortunately respondents tend to restate the obvious in this type of open-ended question.

Q: You gave ABC Company a low rating on the trustworthy scale. What would make them more trustworthy?

A: If they did what they said they would.

This answer provides little concrete information on what should be done to improve trust.

A longer survey or qualitative interview is better for identifying the drivers of trust/mistrust among customers. Is it that the company is not delivering what they said they would (sales reps misleading customers) or does the customer just not believe the company has the capabilities to deliver? This matters when making tactical changes because improving transparency is far different than improving knowledge or capabilities.

The goals of the research and the activities it informs should drive the approach. If the research is intended to monitor key metrics in the market or customer base then a shorter survey makes more sense. If known issues exist but the path forward is unclear, a longer survey or exploratory qualitative would be the best approach.  These approaches can also be used in tandem to good effect (e.g., targeted qualitative research to follow-up on an issue surfaced in a short key metrics survey).

Putting all of the research considerations aside this trust chart provides useful thought exercise about how and why customers and partners should trust us as individuals and our organizations.