Talking to Users Every Week: Scaling Recruitment for User Research

This is the most recent post in our “Product Corner” series. Below, Yesware Product Manager Victoria Grimley details how Yesware’s Product Team adopted the Continuous Discovery framework for accelerating the frequency and impact of their customer conversations.

* * *

Over the last year, Yesware’s product team has adopted the Continuous Discovery framework. This framework rests on the directive of having open-ended conversations with users every week, with the goal of identifying opportunities to alleviate some of their pain points or needs.

When we started to implement this framework, we’ve approached weekly calls with our users with mixed emotions. Our team was excited by the prospect of regular communication with customers and using those conversations to drive our roadmaps, but we felt deeply intimidated by the amount of work that we felt weekly interviews, across each team, demanded.

Where we started

Before implementing Continuous Discovery, the Yesware product and design team was spending time with users in concentrated chunks to validate that an idea met a user need. In order to do this, we followed a set process:

• Gathering and filtering a list of participants from an analytics platform
• Clearing that list with our Customer Success team
• Writing screener questions/cold-emailing your list of participants
• Preparing scripts
• Researching the customer to understand what issues they’ve reported and specific account information
• Actually holding the call and taking notes
• Rewarding the participant with the incentive, if needed
• Marking the participant in some way so we didn’t reach out again
• Synthesizing key findings
• Sharing findings with the team, often creating an accompanying deck

 

What we were finding is that while we were spending a lot of time with users, the full process was taking weeks to complete. Our current process wasn’t scalable to support continuous weekly interviews, but we didn’t know the best way to optimize it.

The Continuous Discovery model encourages cultivating an overall culture of rapid experimentation within the product team and company at large. We applied that principle to our recruitment process and spent months testing new methods of recruiting and interviewing. Ultimately, we found good success, as we’ve interviewed hundreds of users in the last six months. We wanted to share some of our learnings.

Automate when possible

We tested a number of third-party tools in the user recruiting space, such as Respondent.io and Ethn.io. Overall, while some of these tools can be expensive, we found that it streamlined our recruiting process in ways that we couldn’t manually. In one instance, we were able to schedule calls with 4 users who met specific criteria within hours of our initial set up on Respondent.

We also sought to streamline more of the administrative burden. We utilized Yesware’s functionality, such as the Campaigns feature to send personalized emails to a large list of participants and swapped successful email templates and tips that lead to higher connection rates with our users.  We used scheduling tools to allow our customers to quickly book meetings without the back and forth of finding a time that works.

Allow users to self-select

Yesware had existing feedback forms embedded into individual features within the product, where users could write about what they liked or didn’t like. Because we were getting regular feedback through these channels, we added a question to the forms asking if the user was willing to participate in a feedback call with the product team.

Upon receiving a user’s feedback, we would ensure that the relevant product manager would follow up as soon as possible if a user opted into a call. 

We also boosted the calls to action within our product, being more direct with our users that we were developing a feature and wanted them to partner with us to build it.

 

As a result of making the two small tweaks, our calls with our users were richer with insights around the specific feature we were developing. Because they had opted into talking to Yesware, they had tangible and direct feedback on what they liked and didn’t like about the feature. There was little to no need to include a monetary incentive for their time and the legwork to recruit them was dramatically reduced.

Trim away the fat 

Naturally as we began to experiment with the recruitment process, we started to get a better  understanding of which aspects were truly necessary and which weren’t. This will likely look different for your team, but we found that we moved more quickly and interviewed more when we didn’t have to:

• Check the list of participants with our Customer Success (CS) team. Our CS team would make sure we were aware of accounts that didn’t want to participate in user research, but we didn’t have to clear every participant we talked to with them.
• Mark a participant as someone we’ve talked to previously. This obviously only works if you have a larger user base, but we found it was actually pretty rare to talk to a user more than once in a set period of time. We had to be okay with the risk of potentially asking a user to participate in several research interviews at the same time in order to move at the pace we needed to.
• We scaled down the process of sharing and synthesizing findings. It was unlikely that others in our company had time to really dive deep into our longer decks and raw notes, so we questioned whether it was worth the time to be making those artifacts. Instead, we’ve been sharing brief summaries and key takeaways from our research.

Recognize when something doesn’t work and learn

To scale a user research program effectively, we realized we would not only have to be willing to experiment, but we had to be comfortable with the idea that our experiment might not yield the results we wanted. 

We also realized the importance of welcoming open and honest feedback from each team member about what wasn’t working well in their process. Doing so allowed us to practice continuously checking with one another, stay in touch with what our users are saying, and grow stronger as a team.