That’s how Research Recruities 🏆 was born – an award for the most challenging, funniest, or strangest participant recruiting story in the community. We asked people to share their stories and lessons they learned from them, to spread the word about this uncelebrated side of UX research.
To provide a valuable blend of insights we decided to divide the project in two parts. We created a survey and distributed it in the UX communities and among our audience to gather recruiting stories and collect data on what experts consider to be the most detrimental UX research recruiting mistakes.
Apart from that, we reached out to selected UX professionals, known for their contributions to the community, asking them to participate and share their recruiting stories, as well as go-to recruiting tips and strategies.
In this article, you’ll find:
- a shortlist of 5 of the best and most insightful recruiting stories we’ve collected,
- results of the Research Recruities 🏆 survey,
- a list of 7 biggest UX recruiting mistakes + tips to overcome them!
Key Insights
❌ Failing to properly screen the participants is the most common UX research mistake
💥 3 of the most detrimental UX recruiting mistakes are considered to be:
- Not defining clear targeting criteria
- Not screening participants properly
- Over-reliance on convenience sampling
🗓️ The biggest number of UX professionals recruit participants monthly, following closely are the ones who recruit weekly
👥 Most UX professionals recruit for both moderated and unmoderated research, however, a significant number of our respondents recruit only for moderated research
🔎 The median number of participants recruited for a moderated testing study during the year 2023 is 65, while for the unmoderated studies, it’s 157.
➡️ For qualitative insights, scroll down to the 7 Biggest Recruiting Mistakes section, or visit our blog with UX Recruiting Tips from 19 UX Experts
Top 5 Recruiting Stories from Our Survey
Through our survey, we collected 16 great stories. From these, we carefully selected the top 5 stories, forming a shortlist that was then presented to our panel of 13 UX experts (Stephanie Walter, Debbie Levitt, Nikki Anderson-Stanier, Caitlin D. Sullivan, Aneta Kmiecik, Tina Ličková, Julian Della Mattia, Ben Levin, Rianne van de Rijt, Sajani Lokuge, Cory Lebson) to choose the 3 winners.
Each of the experts could pick one story for 1st place (awarded 3 points), 2nd (2 points) and 3rd (1 point). We then summed up the points to create the ranking.
Below, you’ll find all five stories, arranged according to the ranking assigned by our experts.
Note: The stories below are all equally important and full of insights. We didn’t choose to rate them to judge or compare the skills of the professionals involved; our goal was simply to embrace the fun “award” aspect of this initiative and decide on the winners to hand out the prizes fairly.
1st Place: Santa Enters the Study!🎅
Respondent 164, UX Researcher, 5-10 years of experience (wishes to remain anonymous)
When I was coordinating recruiting for a full research team, I worked in an office building where conference rooms had floor to ceiling glass walls – not entirely soundproof, but helpful to be able to see if remote interview participants had arrived.
I strolled over to ensure one participant had made it to his session. This study was looking at retirees, so the participant was in his 60s, with a big, bushy white beard. He was also wearing a bright red shirt.
I was chatting with a colleague for a few minutes outside the conference room while the interview was still going on, when an SVP walked by the — again, not soundproof — conference room. I don’t know what possessed her, but she peeked in, saw the participant on the large flatscreen, cupped her hands around her mouth, leaned in and shouted “SANTA!” Then looked almost bashful and scuttled away.
I nearly fainted.
So much of recruiting is trying to prevent inevitable snafus in logistics or communication, and trying to create a good experience for participants — I never thought this was something I’d need to worry about! Fortunately, I spoke to the research team later and they somehow hadn’t heard the yell, so our participant certainly hadn’t. Phew.
Best part? The participant was, in fact, a part-time mall Santa.
➡️ Lessons learned: Make clear to folks that participant interviews are happening and we need to ensure a quality experience — or create an interview environment where people not directly involved can’t interfere with the process.
2nd Place: Navigating Privacy in Participant Interviews
A story from Andreea Dalia Lazar, Senior UX Researcher, 5-10 years of experience
I was recruiting travelers for a project that I was working on. The interview started with me providing the usual initial instructions (confidentiality and anonymity included). The interview was really insightful and the participant was sharing important details form their traveling experience where they also mentioned their partner.
Everything was going great until the participant’s mood started to drop and they weren’t really eager to answer my questions anymore. When I asked about it, they mentioned they didn’t want anyone to know they have an official partner, and started to explain how complicated their romantic life is.
In the end I reassured the participant that the data is both anonymous and confidential and everything they shared in the interview will not be shared with anyone else. I found this really funny at the time and packed with learnings.
Participants may need reassurance a couple of times on various topics to feel comfortable and safe to share information with us ❤️
➡️ Lessons learned: Participants, like people in general, have very different stories and needs. Always remember to listen more than speak, constantly check for non-verbal cues and make sure they feel comfortable throughout the user experiments. You never know what might be the reason they worry about while sharing their story with you 🙂
Participants, like people in general, have very different stories and needs. Always remember to listen more than speak, constantly check for non-verbal cues and make sure they feel comfortable throughout the user experiments.
3rd Place: “After two days, he threatened to sue me”
Respondent 41, Research Operations Specialist, 10+ years of experience (wishes to remain anonymous)
I once conducted research with a participant who was an IT employee, consented to the research, gave fantastic insights and ended the session amicably.
After two days, he threatened to sue me because he got scared that I might upload his feedback video on social media even though we had stated this will be for internal use only in the consent form.
After a series of calls over next 2 days, I briefly explained to him the purpose of the study all over again and how his content will be used and that I am happy to meet him again outside office to explain in person if that helps. He was convinced and trusted our team from thereon.
➡️ Lessons leaned: Keep your calm and act with wisdom.
——————————————————————————————————————————————–
Honorable mentions👇
Below are the 2 stories from our shortlist that didn’t make the top 3 but are still amazing and full of insight to share, making them totally worth your attention.
4. Tiny Oversight, Big Consequence
A story from Parker Sorensen, Associate Director of Conversion Optimization / UX Research, 5-10 years of experience
I was conducting an unmoderated study for a university within the United States. I set up all my recruiting filters except the country. This resulted in me getting responses from mostly outside the USA.
Not only were their responses not as relevant as they were not our “look alike” audience, but the questions did not make sense to them, as universities in the USA work differently than universities in some other countries.
Outcome: I had to scrap the first run of the test, as the feedback was not helpful. I reran the test, this time with the correct audience filters, and got great feedback.
Learning: Thorough QA of study setup and audience setup is vital to the success of the test. It is worth the time to ensure the study is set up for success. I’m since been more thorough in QA’ing studies prior to launch, and usually launch a pilot sessions before launching all sessions. It can be tedious, but it eliminates the need to scrap and rerun a test!
➡️ Lessons learned: Take the time to 1 – think through who the audience for your study should be, 2 – plan this out in detail (I like to think “how would I filter people out / what would indicate that they are not a good fit for this study”), and finally, 3 – do a thorough QA of your study and audience setup.
Take the time to 1 - think through who the audience for your study should be, 2 - plan this out in detail (I like to think: How would I filter people out? / What would indicate that they are not a good fit for this study?), and finally, 3 - do a thorough QA of your study and audience setup.
5. Diving Deep into Niche Recruitment
A story from Blanche Letakis, Product Manager, <2 years of experience
I was recruiting for a very niche persona, Channel Partner. I knew nothing about this industry and for the duration of this study I was the only researcher on staff. I had to do prep work and industry research.
I started reading blogs about channel partners, watching YouTube videos on channel partners, I just needed to understand who I was going to recruit. My first thought was, because this was such a niche persona I didn’t think the recruiting platforms would be able to actually find participants like this, so I suggested LinkedIn recruiting.
Aaaand because I suggested it, I had to do it. That was a big flop.
After many LinkedIn messages and no responses, I was defeated. So I turned to the recruiting sites, crafted a screener and voilà, I had 2 VERY qualified participants and was able to schedule at least one participant that same week. In fact, my whole recruiting journey took place in just one week.
So even though I failed at my first attempt, I was able to spin it and successfully secure a research session by Friday.
➡️ Lessons learned: When you are recruiting for a niche persona, cast a wide net and try multiple things. But always be focused on your criteria.
Now, let’s dive into the Research Recruities 🏆 Survey and the insights from it. We’ll jump straight into the results but if you’re interested in learning about the demographics of our participants first… ⤵️
Recruiting Stats from our Survey
We wanted to get some statistics and insights what research recruiting looks like in the UX world. We asked the following questions:
- How often do you recruit participants for user research?
- Do you primarily recruit for moderated or unmoderated research, or both?
- How many participants did you recruit for moderated studies last year (2023)?
- How many participants did you recruit for unmoderated studies last year (2023)?
Turns out that the biggest number of our participants (29.6%) recruit monthly. Following closely are the ones who recruit weekly (23.7%), showing a significant commitment as well.
There’s also a smaller segment, 5 people, making up 3.7%, who fall into the ‘Other answer’ category, indicating a variety of less common frequencies, like daily recruiting, or conditional recruiting, depending on the type of the project.
As for the type of study, most of our respondents recruit for both moderated and moderated research (70.4%), while some are recruiting specifically for one of those types, with more people recruiting for moderated research:
According to our data, the median number of participants recruited for a moderated testing study during the year 2023 is 65, while for the unmoderated studies, it’s 157.
To give you some additional context we also analyzed some of the data from the previous reports done on research recruiting.
The annual State of User Research Report, shows that the median number of participants in a moderated study is 8. Those participants are usually recruited with the help of 2 to 3 recruiting methods, depending if researchers are recruiting their own customers or outside participants.
The most popular method for recruiting customers is email, followed by intercept surveys. However, when it comes to recruiting external users, researchers prefer using external recruiting platforms and User Panels, such as the one offered by UXtweak.
7 Biggest Recruiting Mistakes + How to Overcome Them
Apart from gathering remarkable recruiting stories and insights, we wanted to find out what mistakes UX professionals experience and come across most often when it comes to research recruiting. We also asked which ones of them they consider to be the most detrimental.
We asked 2 main questions.
Question #1 – “Which of the following mistakes have you observed or experienced in research recruiting?(select all you experienced)”
With this multiple-choice question we aimed to identify the most common pitfalls in the research recruiting process as observed or experienced by respondents. By providing a list of potential mistakes, the question seeks to gather insights on which areas of the recruiting process are most prone to errors, how widespread these issues are, and to prioritize areas for improvement based on their frequency.
Question #2 – “From the list, which mistake do you consider the most critical or detrimental in research recruiting?”
With that question, we aimed to identify the mistake that participants view as the most severe or having the most significant negative impact on research recruiting. By asking respondents to select the most critical error from the list they were provided in the first question, the aim is to prioritize the issues in terms of their potential damage or disruption to the research process.
The second question was followed by the open-ended question: “How would you handle or overcome this mistake?”, inviting respondents to propose solutions and strategies for addressing the mistake they identified as the most critical.
Here’s what we’ve found out:
According to our data, the most commonly experienced research recruiting mistake is failing to properly screen the participants, which was the option chosen by 17.6 % of our respondents.
Following closely is the mistake of rushing the recruitment process (13.4 %) and not defining clear targeting criteria (10.9%).
The answers to the second question painted a slightly different picture, clearly pointing out the 7 most detrimental UX research recruiting mistakes, according to our participants. Below, we’ll take a look at each of them and their nuances and include tips from our respondents and UX experts for handling those mistakes.
Click on any mistake to jump straight to it, or just keep reading ⬇️
- Not Defining Clear Targeting Criteria
- Not Screening Participants Properly
- Over-reliance on Convenience Sampling
- Ignoring Diversity
- Not Validating Participant Information
- Not Checking for Previous Participation
- Rushing the Recruitment Process
1. Not Defining Clear Targeting Criteria
Most of our respondents (37,46%) chose this mistake as the most detrimental in research recruiting. And it essentially refers to the failure to establish precise and specific criteria for selecting study participants.
This often results in the inclusion of participants who may not embody the characteristics, behaviors, or needs of the intended user group. Such a misalignment can lead to false conclusions and wasted resources on irrelevant research.
This problem often arises when the project scope is broad, or when there’s a rush to start the research without adequate planning. Poor targeting might also stem from a lack of understanding of the user base or the objectives of the study.
👉 How to overcome this mistake?
We asked our respondents how they are handling this issue. Some of the most common answers included:
- clearly defining the goals & hypothesis of the research,
- talking to stakeholders
- taking the time to craft a good screening questionnaire
Here’s a great example of how to NOT approach the process of recruiting a specific user group and what to do instead, from Parker Sorensen, Associate Director of Conversion Optimization:
I believe this comes down to good test planning. I have seen (and in my early days or research did this myself) where researchers recruit a general audience and try to have participants imagine a scenario (ie. Imagine you are looking to purchase a home in the next 1-3 months).
In my experience, it is difficult to impossible for participants to put themselves fully in the scenario - because they are not actually there. A better way is to recruit the exact audience (ie. screen for: Which of the following are you planning to do in the next 1-3 months?, give 5-10 options, and make 'purchase a home' required.
For those at the early stages of their research, here’s a great tip for creating specific targeting criteria from Blanche Letakis, Associate Product Manager & UX Researcher:
I start research studies with a slightly broader target and as sessions go on, I learn more about our target users, and tweak the criteria, including screener surveys.
2. Not Screening Participants Properly
Screening is evidently one of the biggest challenges when it comes to UX research recruiting. Not screening participants properly is also the most widely spread recruiting mistake, according to our data.
This issue usually manifests when there’s either a lack of detailed screening processes or when the screening criteria are not aligned with the study’s objectives.
Failing to conduct effective screening often leads to skewed study results, wasted resources, unqualified or “fake” participants (scammers) entering your study, and other unpleasant consequences that do not contribute to generating meaningful insights.
👉How to overcome this mistake?
Here are some great tips to ensure effective screening:
Andreea Dalia Lazar, Senior UX Researcher, advices pilot testing and peer-reviewing your screeners, as well as checking participants manually if you’re not confident in your screening process yet.
If possible, test before launching to a wider audience, peer-review and opt for manual reviewing of the participants before getting the confidence that the screening process works as it should.
Here’s what Kelly Meredith, Senior Design Researcher, recommends:
Using a research panel that allows UXers to filter for specific criteria, is the first step I try to take. Additionally, we use a variety of screener tactics to help ensure the people signing up or taking our unmoderated tests are the right people.
UX Researcher Alyona Poluiko emphasizes the importance of structuring your questions wisely in order to prevent scammers from entering your tests:
“Ask questions regarding the product but don’t be too straightforward to minimise the amount of “professional respondents” – Alyona Poluiko, UX Researcher
Ask questions regarding the product but don’t be too straightforward to minimise the amount of 'professional respondents'
Nikki Anderson, Founder @ User Research Academy, recommends to craft your screeners with a great attention to detail, a lesson she learned from one of the recruiting experiences at the beginning of her career, which you can read about here.
Think through every piece of information you need from the participant and craft that into a screener question - it might mean the screener is a bit longer (try not to go more than seven questions) but it will enable you to get the right people for your study!
Crafting effective screeners that help you avoid fake participants and help to actually recruit relevant people is a broad topic and we would not have a chance to put all the expert’s tips in a couple of paragraphs.
3. Over-reliance on Convenience Sampling
This mistake refers to the practice of selecting participants based on their ease of access or availability rather than choosing a sample that accurately represents the target user base.
Over-reliance on convenience sampling often happens when decision makers prioritize ease of access and try to save resources on recruiting by finding participants among colleagues, friends and family members.
However, this method usually leads to biased results and insights that do not apply to your target audience.
👉 How to overcome this mistake?
To counter this issue our respondents recommend:
- Defining the recruitment criteria and following it strictly during the recruiting process,
- Including it in your research plan, ensuring everyone has access to it
- Crafting a good screener to eliminate irrelevant people
- Spending more time on the recruiting process, ensuring you have the time to recruit a representative sample
- Growing a sample database of users over time
- Making sure you have appropriate budget
By keeping a database of your previous testers you’re able to recruit them again for another study, without having to invest that much time and energy into recruitment. This is a great alternative to convenience sampling that will actually provide you with valuable data.
💡Pro Tip Did you know you can use UXtweak’s Own Database to build and manage your own participant panel? Import, segment, filter, and easily recruit users for any research studies you run. ➡️ Try it for free now! 
Make sure that you also pay attention to providing accurate incentives, as it may influence the participant’s willingness to participate in your study again. Jasmine (Lo) Winata, Research Operations Specialist, highlights:
Compensate in time! This is something my company is currently struggling with, but what I've found (from my past company) is that when we pay the participants on time, they are more willing to participate again and even refer their network to us for future studies.
Here’s another great tip for growing your database of respondents from Caitlin Sullivan, Product Discovery and User Research Expert:
Ask every participant you find who is a really good fit if they know someone who might be up for joining your research. It can make recruiting the right kind of people much faster!
4. Ignoring Diversity
Diversity here refers not just to demographics like age, gender, or ethnicity, but also to varied user behaviors, experiences, and accessibility needs. When ignoring diversity you’re risking that your product will be only catered to a narrow user base, neglecting the needs of others who didn’t fit the initial participant profile.
Undoubtedly, it’s crucial to remain focused on the specific user group your product or service is intended for. However, it’s also critical to balance the importance of not overlooking diversity, while adhering to your targeting criteria.
👉 How to overcome this mistake?
To ensure you’re looking at a variety of customer types, Kelly Meredith, Senior Design Researcher, recommends:
When looking to talk to a variety of customer types, rather than sending one invite link and hoping for the best, we use a few methods to ensure we are talking to the right mix of customers. We use Calendly routing forms, which have been a huge help! We create a different event type per type of participant and use the routing form (screener) to filter prospective participants to the right event.
For example, we can chat with 5 large customers, 5 small customers, and 5 non-customers in the most efficient and effective way. Before this feature or for those without Calendly Pro, we use Alchemer to create a screener and use the logic to direct potential participants to different sign up links.
5. Not Validating Participant Information
The mistake of not validating participant information can result in fraudulent participants entering the study. Those are scammers and dishonest individuals who trick the screeners and enter the study just to collect the monetary reward. Those people are usually not representativce of your target audience and can significantly skew the results of your research.
👉 How to overcome this mistake?
To counter those and prevent them from ruining your research, Michele Ronsen, Author, Researcher, Educator, Founder, UX Coach, emphasizes, that it’s important to validate participant information before letting them into the study.
Do not allow participants to schedule study sessions directly once they’ve qualified through the screener. Teams of scammers will sometimes re-take screeners until they’ve “passed,” then use those answers to qualify under other additional identities or share the answers with other scammers.
Inviting participants to schedule their sessions takes longer, yet it offers additional benefits. It enables a secondary screening process, provides a chance to confirm the accuracy of the contact information they've provided, and allows you to cross-reference responses to identify potential duplicates among the screeners.
Here’s how Respondent 164, a UX researcher with 5-10 years of experience recommends approaching that problem:
Write better screener questions (no yes/nos, etc.), and when things are tricky - call to verify.
6. Not Checking for Previous Participation
Failing to verify if a participant has previously been involved in similar studies or tests can lead to participant bias and compromised data integrity.
Individuals who have participated in previous studies may have developed certain expectations or biases that could influence their responses and interactions. Additionally, participating in a similar test more than once leads to the person becoming overly familiar with the research process or the product being tested, which can affect their behavior and responses.
Not checking for previous participation means that you probably won’t be able to capture initial reactions and learning processes. This can result in atypical responses that don’t accurately represent the target user base.
👉 How to overcome this mistake?
To avoid this mistake, it’s important to implement processes for tracking and filtering out individuals who have previously participated in related studies. This could involve maintaining a database of past participants or using screening questions to identify prior involvement.
Marek Strba, UX Researcher and Customer Success Lead explains, why in specific cases checking for previous participation is so crucial:
Recruiting participants, who have participated in a similar study or in a previous round of your study doesn’t need to be a problem. In some cases, it is even wanted.
However, you must know whether you are recruiting such participants or not. The “not knowing” is the core of the problem and can lead to unexpected anomalies in the results.
Now, here comes a shameless self-promo.
7. Rushing the Recruitment Process
With tight project timelines and limited budgets, many researchers are facing the issue of stakeholders rushing the recruitment process to meet the schedule. This often leads to Recruiting the first available participants without properly vetting them against the study criteria and overlooking quality screening.
The pool of participants then, ends up being less diverse, often smaller, and full of participants who are not representative of the target user base.
Ironically, rushing recruitment usually leads to inefficiencies, as the need to redo or augment the study due to poor recruitment, which can consume more time and resources in the long run.
👉 How to overcome this mistake?
Jasmine (Lo) Winata, Research Operations Specialist, recommends to tackle this by informing stakeholders and communicating the need for a longer and more structured recruiting process:
Pushback on stakeholders who are rushing recruitment and lay down the realities and expectations regarding recruitment, especially with a new pool (looking for brand new users vs. using a pool we already have access to).
Let them know that recruitment will still happen, but the desired number and quality of users may not always meet expectations.
Biggest Recruiting Struggles and Tips
The last question we asked in the Research Recruities survey, and the one we also posed to the research experts who shared their recruiting stories was:
“Please share any additional tips or strategies that you’ve found effective in user research recruiting.”
The aim was to gather a diverse range of practical insights, tips, and strategies from those with firsthand experience in user research recruiting. By soliciting all the expert tips that we’ve collected, we managed to compile a comprehensive set of best practices for handling some of the most common recruiting struggles.
Survey Demographics
Here comes the boring part – demographics 😅
We asked all our participants to provide some context on their background. We only asked about the details, which we believed could explicitly influence the answers provided to the questions in the core questionnaire.
After cleaning out the data we were left with 135 respondents. The largest group of our respondents were UX Researchers (31%), with UX Designers following closely afterward (20,7%).
Most represented were people with 2-5 years of experience (34.5%). They were closely followed by the people with <2 years of experience and 5-10 years of experience, which was the same amount of respondents (24.1%).
Summing up
To put this all together, Research Recruities was an invaluable opportunity that helped us get the UX community to share their recruiting experiences, offering a unique glimpse into the challenges and successes within the realm of user research recruiting.
It was an honor to read and be able to share all of the amazing recruiting stories, both those funny, emotional, and full of struggles and the lessons UX professionals learned from them.
The mission of Research Recruities was to tell the untold and shed some light on one of the hardest parts of UX research – recruiting, hopefully in an entertaining way. Let us know if we managed that.
What’s next?
If you haven’t already, dive into the world of recruiting stories in this Research Recruities collection – Hilarious & Strange: UXR Participant Recruiting Stories that You Need to Hear
Discover our selection of Research Recruiting Tips from 19 Seasoned UX Experts.
And of course, check out UXtweak! 🐝
We offer multiple ways to make the recruiting process easier for you, including a 155M+ User Panel with quality check, an Own Database feature, which is basically a CRM for managing your participants and inviting them into ANY UXtweak study, and an Onsite Recruiting Widget, that helps to seamlessly recruit your website or app visitors for research studies!
