Meet Caitlin D. Sullivan, a Research and Product Discovery consultant and a new guest of our Women in UX series!
Read along this interview to find more about:
💻Her work approach and daily routine as a freelance consultant
⌚Time-management tips for juggling between multiple projects
📊Tips for handling and presenting controversial research insights
✅Product discovery recommendations for early and mid stage startups
❌Common research mistakes to avoid
Hi Caitlin, tell us a bit about yourself!
I’m a UX Research and Product Discovery consultant working with early and mid stage startups in Europe.
I grew up with a lucky combination of an Anthropologist mother and a Mathematician father, which very much informed how I think about customer research.
I started my career in advertising in the US, where I was exposed to UX designers and researchers for the first time, though this was a pretty new field then in a corporate setting. I then continued working in design agencies in Sweden, where we were essentially paid by clients to be “right”, to find the best possible design solution before launching it into the market in front of millions of people. I’ve taken that thinking into my work with early product teams that need to find the best solution quickly, and before launching untested products into the market.
I’ve worked in three countries now, in three languages, with B2B and B2C/DTC. My biggest passion is helping early product teams implement the right thinking and the right systems to run reliable customer discovery work from the start.
Working as a freelance consultant, time-management tips
Can you tell us a bit more about your current role? What are the main things you’re working on daily as a Product Discovery Consultant?
As a consultant, I work with multiple clients at a time. It’s a lot of juggling and time management!
My daily patterns shift a lot depending on the week and the specific projects. Last week, I spent every morning teaching one startup how to run experiments, getting them from zero experience to launching their first experiment. In the afternoons, I ran interviews for another client, working to deeply understand what events, circumstances and feelings drove their customers to cancel.
Sometimes, I’m planning and executing research projects myself. More often, I’m supporting teams to run their own UX research and experiments. I prefer the second, because I want teams to succeed at understanding their customers in the future without needing me or another consultant to do that in the future.
In any project or advising role I take on, I have to play the role of educator. I typically help early stage teams with low research maturity. From my perspective, part of my job is to help them understand foundational concepts like risk, making better bets, and how to document insights properly for use over time. I drill these concepts because I want every team I work with to become better decision-makers when I leave.
I want every team I work with to become better decision-makers when I leave.
As a freelancer, how do you manage to juggle between different topics and industries when it comes to research?
I have to be very organized. There’s a heavy reliance on having the right systems in place for myself, as well as for clients.
To give others hope, I’m not naturally a very organized person. But I have trained myself well over the years. I have tools and systems in place to keep track of everything I learn for myself and clients.
I’m obsessive about time-blocking my days. I have certain days or hours available for certain clients. If I can help it, I won’t schedule really important work or presentations for two different clients in one day. It’s too much cognitive load to switch my brain from, for example, thinking deeply about pet insurance customer needs to planning experimentation processes for a SaaS team serving solopreneurs.
Within those blocked client hours, I split time into team meeting hours and deep work hours. In those deep work hours, I pick the one thing I need to accomplish that day, and focus on moving it forward.
Secondly, I’ve handled research projects and trained others for years now, so I’ve built up my template collections and systems that I’ve tested many times.
Having systems and templates makes many research and training steps go much faster. In the end, I’m left with more time to focus on important steps like analysis. I think when I’ve been less organized and more rushed, I unfortunately sped through research analysis faster than I should have.
I try to teach others, too, to automate as much of the research process as possible and leave more time for thorough analysis and discussion.
Could you share some strategies that help you keep track and manage multiple client research projects simultaneously?
My go-to tools at the moment are all templates I’ve built for myself in Notion. I have templates to keep track of everything I need for each client: project tracking and to-do’s, meeting notes and stakeholder needs, feedback, research plans, education documents, and methods for tracking the impact of research projects.
For example, let’s say I need to start training workshops with two clients in a week. This same week, I’m finishing up a research project where I’m carrying out interviews. For all of these projects, I will rely heavily on templates from work I’ve done in the past.
The two training workshops may overlap in content slightly, and I’ll reuse any content I can. I’ll plan the interviews I’m leading with templates from previous, similar projects I’ve run.
Some of the templates are for me, and some are templates I give to clients so they can jump into research and immediately run, not walk.
From a time-management side, a specific tactic that helps a lot is to write each day’s to-do list the day before. Before I close a client’s Slack, email and calendar on a Monday, I write down the things I need to do for them on my next day with them. I prioritize this list, I kill about 50% of what’s on it. I choose one item I need to finish on that day for my day to be over and successful. Anything else goes on my “not a crisis” list.
My list looks like this:
Priority 1: [what I need to finish on this day]
- Finish all materials for experimentation workshop Days 1-2 for Client A
Not a crisis if I don’t do these (but nice if I manage):
- Start Workshop Day 3 materials for Client A
- Send Hypothesis worksheet to Client A
- Remind Client B to get me a Mixpanel login
Based on your experience, what key skills and mindsets should new UX researchers cultivate to thrive in a freelance or consultancy setting?
Great question! Coming from previously working in-house as a full-time employee, the biggest challenge is often having to be the one deciding what to do. If you’ve never managed a team before, this may be new.
Clients turn to me for advice, and sometimes need support deciding which research to prioritize. I’m not usually hired by a company that has one single clear project they want me to run for them. They need guidance to think about research clearly and decide where to focus. I often help them decide what projects I should be used for.
There’s also a part of freelancing / consulting that a lot of people try to avoid. We have to be salespeople. I’ve luckily had clients come to me by referral so far. But I also spent years building a network that now delivers clients to me. In those years, I think I marketed myself in a way that helps me now. It’s not fun for most of us to think about ourselves as needing to sell. But it’s important as an independent consultant to think about how you present yourself to the world, and what kind of clients you position yourself for.
Dealing with controversial insights in research
You mentioned you wanted to specifically talk about the topic of handling controversial findings from research, do you often face the problem of teams ignoring unpopular research insights? If so, what do you think is the reason for that?
It’s a common story: teams with lower research maturity often want better insights. But they’re also afraid to face insights that tell a different story than they’ve been telling themselves.
Passionate founders can be guilty of ignoring insights that go against their vision.
I’ve mentored multiple UX Researchers who asked me for support with this specifically. When a first UXR joins a team that never did research before, there can be a blocker to insights that challenge what the team believes. The founders or Product Managers may have a fixed idea about who the customer is and what they need, based on understanding gathered from a single point in time.
How do you handle such problems? Based on your experience, are there any tips to get teams to pay attention and act on those insights?
When a team finds insights that challenge strongly held beliefs internally, it’s important to make a good case and document the case for future review.
If I’ve heard something controversial in just three interviews, I won’t make a big deal about it until I have more, consistent proof. I’ll check if data tells a similar story. I’ll interview a few more of the same type of user to compare. I want to make a strong case before I push management to consider something drastically different from their beliefs.
If the C-level or a PM makes a decision that is clearly counter to the story that data and customer stories tell, this should be documented. There should be some accountability and tracking between insights and decisions made.
Every team should aim to learn from decisions that used and ignored insights over time. I’ll simply ask: “Did this decision align with what the insights told us?” “And did our product [change/adjustment/addition] accomplish what we intended – yes or no?”.
Any product change that fails to solve a customer problem, and ignores insights, needs to be noted somewhere and learned from. Otherwise, we won’t improve over time. I don’t want people feeling bad about this or personally targeted. No one needs to put their individual name on an uninformed decision. I want teams to learn to be more objective, and to make better decisions over time.
Product discovery and researching for product-market fit
What are some of the most valuable lessons you’ve learned from working with early and mid stage startups? What do you find to be the most interesting about that niche?
I love early stage companies because they’re messy! I get to step in and show them that, even in the early days, we can create systems and improve product decisions faster than they thought we could.
There’s a lot of pushback against research here, though. Many early stage teams think their founders know exactly who the customer is, so they don’t need research. Or they prioritize shipping over understanding customer needs. That sometimes works for a while. Eventually, they hit a point where it doesn’t work anymore. Then they have no idea how to fix the product before they run out of cash, because they don’t know their customers.
In contrast, big tech companies are so structured and regimented, and run a ton of very professional, mature research. But I often hear from friends working at the big names that there isn’t much room for trying new things around their protocols and politics. That’s not a world I find fun. I like to find the best balance between rigor and speed. I like working with teams where impact is visible quite soon after starting, and I don’t need to pitch a new method through five levels of management over months.
I work with early stage teams because I learn most here. Early stage startups have so much time pressure that I am frequently looking for even faster ways to understand a customer without compromising a decent level of rigor. I learn constantly, because they push me to try new approaches all the time.
What research methodologies do you find to be the most effective for product discovery? What are the main strategies you use to make sure clients are always “right” with their design solutions?
It depends on whether a team has a product already or not, and has a foundational understanding of their core audience or not.
Generally, I always encourage interviews. I think you can gain 70% of what you need to know from interviews, if you do them well. There are also different segments, and customers at different journey stages, that you should understand if you have a live product and a short runway. Most teams that have done interviews focus heavily on understanding acquisition. They ignore people who just canceled, for example, who can give us a goldmine of information about product value and missed opportunities.
Generally, I always encourage interviews. I think you can gain 70% of what you need to know from interviews, if you do them well.
Some things should be tested to observe live behavior. Demand testing and usability testing can be very important to understand how a customer understands and reacts to the product. I often teach teams to run landing page tests as a simple way to test big assumptions that will make or break your product launch.
I also encourage small tests with existing customers when a client feels comfortable with this. For example, we can test a new idea in an email newsletter or one small extra module on a home or dashboard. You can then quickly see how strong or weak demand is for an idea you had. I always encourage teams to ask, “What is the smallest possible test that gets us a clear “support” or “reject” for our most important hypothesis?”.
Given your experience in both industries, do you believe the approach to customer discovery varies significantly between B2B and B2C/DTC? If so, could you explain the key differences?
I don’t think implementing research is so different between B2B and B2C. But there are two main differences I see.
In my experience, the biggest difference lies in recruiting. Recruiting B2B audiences can be very challenging, and more time-consuming. I’ve interviewed tough audiences like chain restaurant owners, dog breeders, and owners of private medical practices. These audiences are much harder to find, reach and convince to join research than, say, consumers who stream music and podcasts.
It can also be crucial to do more desk research about an industry before B2B customer sessions. I’ve experienced how much better research results are when I first thoroughly understood the context that my client’s B2B customers operated in. When you understand their context better, you can ask more targeted questions or create more realistic tasks that generate more concrete customer feedback.
It can certainly be helpful to understand a B2C customer’s context ahead of time. But there can be more prep work ahead of B2B research to understand a new industry or topic and ask the right questions.
Can you share some of your go-to tips for researching for product-market fit?
I’ve used a simple survey for product market fit quite often, referred to as the Sean Ellis Test.
You ask customers, “How would you feel if you could no longer use [product]?”
The possible responses are, ‘not disappointed,’ ‘somewhat disappointed,’ and ‘very disappointed’.
Based on their responses, you get a measure of how strongly attached to your product customers are. Ideally, >40% of respondents say they would be “very disappointed” without your product. That’s a signal that your product delivers for a real need in the market.
Product market fit isn’t something you can perfectly pinpoint. But I’ve seen this test give results that clearly aligned with the sense of fit vs. struggle I was seeing a company deal with.
I ran this survey for two clients at the same time a while back. Both had products that were a few years old, and retention was always a struggle. They both felt like they were pushing a boulder up Everest to retain users. We looked at various metrics to get a sense of what was happening. CAC was high, customer referrals were low, etc. There were various signs of struggle in the data. Product changes for optimization weren’t moving the needle for them at all.
The Sean Ellis test results showed that >50% would be “somewhat disappointed” if the product disappeared. Around 25% would be “very disappointed.” This meant that there were far fewer very attached customers than those who could live without the product.
From there, I recruited people from the “very disappointed” group and the “somewhat disappointed” group to dig deeper. I focused on identifying the problems that the product solved well for those who couldn’t live without it. Then I dug for the problems that remained unsolved for the people who could live without it, and so on.
In both cases, these teams had not reached a good level of product market fit. They didn’t know what they still needed to solve in the product to move people from “somewhat disappointed” to “very disappointed” without it.
I recommend teams to go through this process, make a product improvement based on interview findings, then repeat the survey to re-measure. This benchmarking can make it much clearer whether product updates measurably improved the customer’s perceived value of the product over time. If the percent improves, it’s a sign you’re moving toward a better fit in the market.
What common mistakes do you see teams make when researching for product-market fit, and how can they be avoided?
I don’t see many teams trying to understand their own fit in the market more tangibly and measurably, to be honest. Many assume that product market fit isn’t something we can measure and track.
There are a handful of tactics and measures to visualize how well you’ve solved a market problem. I rarely meet a founder who has tried any of them.
I think the common startup mistake is pushing acquisition to improve numbers for potential investment rounds, without taking time to identify problems that led to high acquisition cost and retention struggles. Many teams don’t want to do that work, or they don’t think about it. In the end, they actually work harder pushing that boulder up Everest than they would have if they spoke to more customers.
The common startup mistake is pushing acquisition to improve numbers for potential investment rounds, without taking time to identify problems that led to high acquisition cost and retention struggles.
Women in UX
What do you think is the best part of being a woman in the UX/tech industry?
I’ve felt very lucky to have a lot of supportive women around me in my career so far. Often, it was a matter of sticking together as women in a still male-dominated world. Even if it comes from a challenging place, I think we’re all making the best of it when we support each other, online and off. The supportive environment between women in tech, when it happens, is remarkable and inspiring, and I want to see it even more.
What is your message to other people in the UX industry?
Good design does not exist without a deep understanding of what customers need from us. There is no “business” without solving customer problems. I don’t care what your job title is, you will do your job better when you spend time with customers. I believe anyone can learn how to talk with customers, and do it well.
