Bias lives in the execution, not the approach

Sometimes in the buzz over new research tools the practitioners and media are overzealous in identifying insights that the new tool provides that old tools miss. In reality the problem typically isn’t with the old tools, it is with how they are used. But it’s always easier and more interesting to grab onto the next new thing than it is to address the misuse of existing tools and process.

Anthropological studies currently receive a lot of attention. We agree that they are a great approach and should be used, when appropriate, by companies with the resources to do so. However, they are not the only way to delve into the underlying motivations of customers. You would not guess this by the way Christian Madsbjerg and Mikkel Rasmussen describe the benefits of the approach in March issue of the Harvard Business Review. It profiles four companies that used anthropologists to help better understand their customer base. The companies included Lego and medical device manufacturer Coloplast. The problems the article outlines relate to businesses that had an inward focus on products or strategy and had lost sight of their customer’s needs. Lego lost its way as it expanded into new areas such as action figures and new product lines in an attempt to appeal to a broader market base. Coloplast focused its efforts and research on improving specific functionality of its personal care medical devices without considering the entire person and how their devices are used in the real world.

In framing the subsequent research the authors make a statement that we couldn’t agree with more: “The conceptual shift requires companies to stop looking at the market, product, and the customer from their own perspective and examine the customer’s perspective instead.” This should be the objective of any research study from surveys to social anthropology. Observing people in their natural habitat is an excellent way to understand how they think and act. But it’s not the only way, especially in product categories where people can reliable describe their behaviors, business processes, etc. One of the main differences between traditional approaches that we use vs. observational studies is asking people what they do/think versus watching people do what they do. Here are two B2B examples where traditional research uncovered the human drivers of decisions.

Online collaboration tools: In the early days of online collaboration tools many of the providers were engaged in a features and functionality arms race. The general feeling was the way to get people to use the tools was to increase the amount of things that they could do with them. In addition, as soon as one provider added a functionality everyone else raced to add it as well. We worked with one of the providers and helped them take a step back and think about the product from the customer’s perspective.  We conducted a series of qualitative in-depth interviews with professionals that work with colleagues in different locations. Our questions focused on the process of working together on a project, not the specific activities that could be provided by an online collaboration tool. We found a disconnect between how people actually work together and the functionality the online tools were pushing. People want to maintain ownership and control over their parts of a project even when collaborating with others. When working together in-person they didn’t want to open a laptop and start working on a document in real time with their colleagues. They preferred to exchange ideas, take those ideas back to their office, and incorporate those ideas into their next round of revisions and then send out a new draft for review. Therefore, even though online collaboration tools made it easy for them to work on the same document at the same time with a colleague they didn’t necessarily want to use this capability. This insight came from asking them about how they work together with their peers. We would have likely gotten very different answers if we are focused the conversation on the features and functionality of collaboration tools themselves.

BI and analytics: Despite their potential benefits the market for BI and analytics tools did not grow as fast as most vendors and pundits expected. On behalf of a client in the space we explored the barriers to the adoption of the tools by conducting a series of in-depth interviews. On the surface we found that most companies were relatively happy with the current tools and processes. If we left it at that the research would be guilty of charges leveled by the anthropologists. Fortunately we dug deeper and asked about how things were done, by whom, their understanding and perceptions of the process, etc. Two key insights emerged. The teams that would benefit the most form analytics tools had been using the same processes for so long that they had lost sight of how cumbersome and inefficient their processes actually were. Senior management didn’t have any real view into their organization’s ability to run custom reports because whatever they asked for showed up on their desk. They didn’t know if one person was using a reporting tool or if five people were working feverishly to create the report manually.

A key driver of the success of both these studies was that the client had an open mind and wanted to explore the products from the customers’ point of view, rather than testing if a specific feature or improvement would compel the market to purchase its products. We used traditional in-depth interviews to conduct the research. An alternative approach in the first case would have been to have anthropologists follow and watch people interact in meetings and when working on documents. In the second study, they could have sat in with the business analysts to see how they go about addressing requests for information by senior management. The anthropologists may have gained a few more insights than the traditional approach. However, the traditional approach identified the big issues and likely did so at a far lower cost and shorter timeline than using a team of anthropologists.

Some would argue that because traditional research approaches work best when exploring existing hypotheses they are more susceptible to confirmation biases and more likely to overlook the unknown, unknowns. We would counter that biases stem primarily from those designing the research and interpreting the findings – not the research tools themselves. This leads to a key element that is often left out of discussions about research tools and techniques. Any approach is only as good as the people executing the research and conducting the analysis. Just as good management will always trump the latest management technique (Six Sigma, TQM, LEAN, etc.), a good research team will usually trump the most sophisticated research tool.

As with all tools that help companies gain greater insights into their customers, we believe that anthropologic studies have a place in the researcher’s tool box and that making space for a new tool does not mean the value of the existing tools diminishes.