Using and selecting a research tool well is impressive, but designing a research study which incorporates a mix of tools makes your insights far more powerful.
To put it bluntly, insights observed in two or more research methods provide a higher degree of confidence than those observed in only one. As a UX research or design professional, employing Mixed Method Research could be a leap forward in the credibility and implementation of the insights you uncover.
This post is for anyone working in or studying UX who wants to improve their research by using a Mixed Methods approach, and covers the topic from the basics up to the design of a study.
Table of contents
Key Terms in Mixed Method Research
For those looking for a quick leg up, simply understanding the following terms will get you a long way towards the knowledge you need to use Mixed Method UX Research.
What is Mixed Method Research?
In research, method means the general approach to be taken to the topic. Mixed Method Research, sometimes called Hybrid Method Research, combines two or more methods in a single study. By observing and assessing a topic from different approaches, we expect to uncover more meaningful and potentially impactful insights.
Qualitative and Quantitative Methods
Essentially, there are two kinds of methods, and Mixed Method Research studies use at least one of each kind.
Qualitative methods are those which are more personal or which bring to light attitudes, behaviours and thought patterns. Examples are things like interviews and user workshops.
Quantitative methods are those which use measured data that can be expressed numerically. Examples would be surveys and big data pools.
Interestingly, it is sometimes possible to extract numeric, quantitative data from qualitative research, which can be helpful if a study is particularly constrained.
What is multi-method research?
A pro tip, though a bit pedantic, is that multi-method research is different to mixed method research. Multi-method is the art of conducting two or more studies separately but on the same topic. However, Mixed method is conducting one single study that engages several methods, and is the topic of this article.
Why use Mixed Method Research?
Mixed Method Research is increasingly important
Over the last decade, a growing volume of successful product strategy and design has aimed to satisfy end-users. This hinges on deep, insightful research, and furthermore, end-user research informs how the product is marketed and sold. A Mixed Method approach provides deep, rich research insights, which are needed as more businesses turn to a product-led-growth strategy.
Mixed Method Research has many advantages
Your research could benefit from Mixed Method Research by being more:
- Confident: Present an impressive, whole picture, which is the kind of information most stakeholders appreciate.
- Efficient: Combining methods can have overlap, which can reduce time and money costs. For example, a survey can be used as the first step in an interview, cutting down the number of interview questions, while allowing the survey to only address quantitative questions.
- Adaptable: because the parts of a Mixed Method study can be conducted sequentially, you can change the nature of the study relative to new observations. Like structure? That’s fine too, a Mixed Method study needs to be planned out.
- Collaborative: Each part of a Mixed Method study is potentially a discrete study, so the whole project can be broken down into smaller parts which can be undertaken by different people or teams. If you only have limited people or time, you can still design a Mixed Method Research study within the resources available.
- Broad: More complicated and pivotal topics are better approached with a wider understanding and therefore Mixed Method Research.
In summary, the right Mixed Method Research study will present authoritative findings, giving your stakeholders confidence, and your work the best chance of making a positive impact.
When should I use Mixed Method Research?
If you have the scope available, a Mixed Method approach will always create more value than a single method study. In particular, if the topic is particularly complicated (such as ‘how will users onboard our product?’), or of a strategic nature (such as ‘how should we offer file storage for users?’), it’s almost essential to use a Mixed Method Research approach.
However, if your topic is a simple choice, such as ‘which colour is easier to read?’, broadness is not helpful, and Mixed Method Research is probably overkill.
Do I need to be an expert, or have lots of time or money?
No!
If you have time, money, knowledge, or any other constraints, you can still design a study within your reach, which combines methods. Designing the study is up to you, and there is no strict requirement, other than using both qual and quant methods. So design a study that suits the resources available.
Also, each component study, and the tasks within each study, can be broken down and collaborated on, so the researcher can engage with managers, designers, engineers and other researchers. Breaking down a study will create smaller, more manageable tasks, and collaborating will help fill in your knowledge gaps.
How to design a Mixed Method Research study
The following structure can be used to design a Mixed Methods Research study.
Scope
- Definition of the area of study
- Study particulars, like timeframe, location, tools, people
- How will rigour be ensured in each method and in the study as a whole?
Methods
- Methods to be used
- Procedure for each method
- Aims for each method
- Resources required for each method
Outline
- Flow chart, arranging the methods as a process
- Artefacts to be created to communicate the findings at the end
Common quantitative research methods
Surveys – to measure frequencies and attitudes using rating scales (Semantic Differentials, Likert Scale, etc.); most easily conducted with online tools like Google Forms.
Analytics – website and social media data collected from your own internal properties, or from publicly available resources.
Quantitative User Testing – collect metrics by observing how individuals perform realistic tasks on the relevant product or prototype. Tools like UX Tweak can help here.
A/B Testing – present one of two (or more) concepts to each user and collect metric data on user goals. This may be on a live product with real users, or with a more select group, however to be quantitatively rigorous, the user pool should be as large as possible – again, UX Tweak is your friend.
Statistics – access census data, or other online large data tools to gather macro trend insights.
Common qualitative research methods
Interviews – participants will give their purported attitudes, perceptions and behaviours. Look out for the stories and particular language used, which may uncover profound insights.
Diaries – for considered, potentially longer responses. Encourage participants to include photos, videos, drawings, and sound recordings to describe their experiences.
Observations of environments and behaviours – can uncover things which are not mentioned in interviews and diaries. Ideally, videos and photos are taken for thorough assessment. These can also be conducted with tools like UX Tweak.
Focus groups – bring together a small group of participants to uncover perspectives and optionally generate artefacts to illustrate their views.
Co-creation – involve one or two of your managers, designers or engineers in a focus group, with the intent of understanding user problems and optionally sketching out potential solutions.
Learn more about the differences between Qualitative and Quantitative UX Research.
What are the different approaches for designing a Mixed Method study?
Although you’ll create a MIxed Method study design which suits your individual goals, there are a few general classifications which studies fall into, and are helpful to know.
Triangulated Mixed Method Research design
The qualitative and quantitative methods are conducted separately but at the same time and with equal weighting given to the outcomes. Triangulated is therefore a ‘single phase’ design approach.
Example: on the study day, one researcher interviews participants one-at-a-time while a second researcher oversees the completion of a quantitative survey by each participant.
Good for: when you have a smaller budget of time, money and people; or where disparity between the results is acceptable. Triangulated may therefore be best at the start of a larger project to gain general understanding which can be further tested throughout the project.
Exploratory Mixed Method Research design
In the first phase, a qualitative method is used. In the second phase, quantitative methods are used to validate the initial qualitative results.
Example: a focus group discusses their perspectives on an app. The key insights from the focus group are then compared with actual analytics from the app.
Good for: when the topic would benefit from innovative thinking; when the audience is small; or when you expect the participants to be reluctant with sharing their views.
Explanatory Mixed Method Research design
In the first phase, a quantitative method is used. Qualitative methods are then used to help create a more full picture of the area of study.
Example: a user survey indicated dislike for a certain website feature. User observation was then used to determine how easy it was to use that feature on the home page.
Good for: when the topic is strategic to business, especially in large, rapidly changing markets.
Iterative Mixed Method Research design
Iterative, dynamic or interactive Mixed Method studies are designed to adapt to new information as it becomes uncovered through research. It can start with either qualitative or quantitative methods. As each phase is completed, the researcher uses the results to form an approach to the next phase. It may have several phases and flip between qualitative and quantitative as needed.
Example: user interviews uncovered discontent with the settings in an app. So the researcher led a co-creation workshop with a user group and product manager. The co-creation uncovered that the issue was specifically about location settings. This was validated by app analytics which showed that most users could not find location settings.
Good for: when you have the time and confidence to shift approach as you go; for larger projects.
Shortcut, minor and embedded study Mixed Method Research design
There are simple ways to use Mixed Method when working with limited resources, or when seeking reassurance without a detailed study. Similarly, ‘embedded’ designs have a primary data set which is the focus of the study, but use a secondary set in a different method for comparison.
Shortcut qualitative methods include: asking the people around you to share their perspectives; keeping a careful diary of your own behaviours; assessing social media and review comments; and covertly observing users in the wild (respecting privacy, of course!).
Shortcut quantitative methods include: conducting a limited survey or test with online user testing platforms; accessing existing web analytics for your business or found online; and assessing social media and review reaction statistics.
You are ready to start!
Like with many things, your Mixed Method Research will be greatly enhanced through practice. If you’re already using a method of research, then combining it with another can be a simple way to add validation and confidence to your results. If you’re just starting out with research, you might be surprised by how powerful even the most basic Mixed Method Research study can be.
