Greenbook-Forsta partnership bolsters validity and integrity of GRIT Report

How Forsta helped Greenbook to execute complex survey of market research professionals

Introduction

If your work involves research, analytics, or insights, you’ve probably heard of the GRIT Report and may have seen the recently released Business & Innovation edition, the first product of the new Greenbook-Forsta partnership. Read on for an inside look at how this partnership helped push GRIT to a new level.

Since being founded in 1962, Greenbook has become a leading voice in the market research and consumer insights space for both B2B and B2C audiences. With a mission to move the insights industry forward by promoting innovation, Greenbook inspires, informs, and connects insights professionals across the globe through its online directory, webinars, articles, reports, and the IIEX global event series.

For over two decades, Greenbook Research Industry Trends (GRIT) Reports have provided a comprehensive fact base of important trends in the insights, analytics, and research industry. The information and analyses in these reports help brands to grow their insights practices, suppliers to grow their revenue, and insights professionals to grow their careers.

Nelson Whipple, GRIT Research Director since 2019, started working on the report in 2018. Responsible for planning content, writing the survey, cleaning and analyzing the data, writing the report, and coordinating publication, Nelson entered the partnership with Forsta clear on what he needed from a market research technology partner.

The challenge

A diverse topic set and wide ranging sample group offers challenges to controlling size and composition.

Lenny Murphy, Greenbook’s Chief Advisor for Insights & Development, had a dream to provide a resource “for the industry, to understand the industry.” In that spirit, recruiting for GRIT has always “followed the industry” by enabling Greenbook contacts to invite their peers and by recruiting via any social media platform likely to be frequented by research and insights professionals. In this way, GRIT sample composition represents the industry as it evolves, rather than through any preconceived idea of who belongs. However, such an approach makes it difficult to control the sample size and composition.

GRIT Reports cover a wide range of topics, and this also comes with challenges. For example, it forces a level of generality which may unnerve some subject matter experts who expect more granularity. With a comprehensive volume of sections to process and digest, Greenbook needed a technology partner who could collect a large amount of data and analyze a wide range of topics across a diverse group of respondents. Strong analytic capabilities are essential to ensure the data provide valuable and accurate insights.

Greenbook was also in a bit of a time crunch because it had not approached Forsta until late spring, but still needed to complete fieldwork before summer got into full swing for much of the world. This situation placed a premium on speed, and on communication and commitment to deadlines.

And although Forsta’s technology was the best fit for the job, any new tech comes with its own learning curve. The team didn’t have the luxury of time to get up to speed with this technology, especially as much of the target audience might check out of the office soon to enjoy the summer, rendering them unavailable.

---

The solution

Forsta’s technology is built for market research professionals who want to deliver insightful, and impactful research.

While the process Forsta followed didn’t significantly differ from Greenbook’s prior surveys, Greenbook was impressed by our level of proficiency and told us that it set Forsta apart. With a comprehensive understanding of both the research process and the purpose behind it, our technology is built to anticipate and meet the needs of organizations that want to execute highly professional work that will stand up to scrutiny. We know what elements are necessary to produce research that works, and Greenbook reinforced how important this was to the all-around success of the project.

On the technology side, we kept an accurate count of completes and file progress, which may seem trivial, but not to Greenbook. For example, in their experience, some platforms keep “accurate counts,” but not relevant ones because they tend to publish idiosyncratic metrics that are not what the researcher needs to manage the project.

Initially hesitant to use our knowledge base due to expectations that it would be incomplete, difficult to use, or otherwise not address the questions they had, Greenbook found the opposite to be true. They said our knowledge base surrounding the platform was far more detailed than they expected, but even in rare cases when an answer could not be found, our team was able to answer all questions fully and accurately because we understood why these questions were relevant, and, as Greenbook noted, we were happy to do so.

Our services team’s market research expertise made things especially easy for Greenbook. “The timeline was much shorter than I expected, but there was no tension around meeting it or not meeting it once it was set”, said Nelson.

Forsta was also instrumental in helping Greenbook understand “empty cases” and where they came from – those data records that appear in the database but contain no information, except that the landing page was accessed. In their experience, these always occur with any platform (and get averaged into the survey duration metric!) and occur in great numbers, but no other company had been able to explain it. As this phenomenon could indicate attempts at survey fraud or some problem with respondent engagement, Greenbook needed technology – and experience – they could rely on, and Forsta provided answers.

Crucially, Forsta’s research technology allows researchers to analyze data using cross tabs and gives users the ability to create different sets or reports via the comprehensive and easy to use filters and banners we enable them to construct. The platform is specifically designed to meet the needs of the research professional.

When it comes to the learning curve associated with new technology, Forsta’s Services team was on hand every step of the way to help set up the team and the project for success from the start. Our Services team are available to answer questions and maintain an extensive Knowledge Base aimed at empowering clients with the information they need to achieve their goals.

The outcome

In partnership with Greenbook, Forsta successfully:

• Provided comprehensive questionnaire consultation and guidance on complex survey programming logic, whilst calling out areas for further improvements and outdated survey instructions.
• Applied best practice survey ‘look and feel’ and customized questions for an improved user experience.
• Utilized its Sawtooth capabilities to generate a Max Diff design file and programmed it into the survey

Paying close attention to survey quality and validity, we deployed our standard data cleaning tools and procedures to increase survey completion rates, which were especially important given the open and flexible nature of the recruiting methodology.

As do we all, Greenbook receives a high number of fake surveys, many of which appear to be AI-assisted, and Forsta’s tools and procedures were critical to catching fraudulent surveys – ensuring the integrity of the data used  and the validity of the final GRIT Report. Forsta and Greenbook joined forces to take a closer look at the data to identify fraud and exclude those fraudulent survey results. Forsta’s technology is designed to offer this required visibility.

Forsta technology offers qc flags and an overall assessment; the user can either take those flags or make their own assessment based on the flags, depending on how much they want to control in each case. With tools built into our platform to flag bad or suspicious surveys, and the professional experience to instantly detect the difference between good and bad surveys, we were able to prevent bad surveys from appearing in Greenbook’s final datasets.

---

“Forsta offered a true partnership, and they have extensive market research experience. So, when I came with questions, they understood them and answered them…Additionally, the Knowledge Base had “more detail and answered more questions than I expected.”

Nelson Whipple, GRIT Research Director at Greenbook

The future

Based on their successful experience with the GRIT Business & Innovation Report, Greenbook and Forsta are partnering for the imminent GRIT Insights Practice survey which explores how insights professionals conduct their work, including the methodologies and tools they use and how they spend their time. Due to the quick run-up to the B&I survey, the partners did not have a chance to explore some features in detail. Greenbook looks forward to exploring Forsta’s capabilities more deeply and discussing other ideas Forsta may have for dealing with the nuances for the GRIT approach and improving the experience for everyone involved.