Key Takeaways
💡Cognitive walkthroughs help evaluate how new users explore and understand products, leading to better user experiences and smarter design decisions.
⌚ This method is ideal for early-stage testing with low-fidelity prototypes, identifying issues before they become costly.
🚩 By focusing on user flows and cognitive steps, teams can uncover usability problems and refine product learnability.
🧑🏻🤝🧑🏽 Although the cognitive walkthrough method is tailored to internal participants, the essential concepts can also be adapted for larger, external group testing.
What is a cognitive walkthrough?
Cognitive walkthroughs are a quick and reliable way to evaluate new or complex flows, offering a structured approach for internal discussions and flexibility for external testing, making it easier to create user-friendly, efficient products.
Reviewers pretend to be users and try to complete various tasks. They want to see if the website or app is user-friendly and if new users can easily figure out how to use it.
The goal is to make sure that the interface is intuitive and accessible for everyone.
Why you should conduct a cognitive walkthrough
- Adds structure and objectivity: cognitive walkthroughs bring rigor to discussions, reducing the influence of opinions or preferences and creating a more consistent, objective evaluation process.
- Encourages Fresh Perspectives: by stepping back from routine tasks, participants can critically evaluate the product, uncover new ideas, and identify improvements that may be overlooked during daily project work.
Pro Tip 💡
When conducting a cognitive walkthrough, include different team members to get a variety of perspectives.
- Early issue detection: by identifying usability issues early, this method helps prevent costly adjustments later in the development cycle.
- Quick and low-cost: ideal for early-stage testing, cognitive walkthroughs are a fast and budget-friendly way to assess whether a new product or feature supports user goals effectively.
- Supports thoughtful development: these walkthroughs provide a strong foundation for your project, helping teams align on user needs and optimal workflows from the start.
- Works with rough prototypes: this method is well-suited for early concepts, allowing teams to validate ideas and workflows without polished designs.
- Forgiving audience: internal participants, who typically form the testing group, are more understanding of rough concepts, allowing for constructive product feedback on functionality rather than design.
- Small, Focused Groups: walkthroughs are conducted with a small team, including a presenter who can clarify missing elements in the prototype, keeping the process streamlined.
How to conduct a cognitive walkthrough
- Define objectives: begin by determining the specific tasks or actions you want to evaluate and the aspects of usability you aim to assess.
- Create user personas: develop hypothetical user personas based on your target audience. These personas should reflect their characteristics, goals, and prior knowledge to guide the walkthrough effectively.
- Define user tasks: clearly define the tasks users are expected to perform with the product. These tasks should be specific and representative of typical user interactions.
- Develop user action sequences: for each task, outline a logical sequence of steps that a user would need to take to complete the task successfully. This sequence forms the basis of the walkthrough.
- Perform the walkthrough: during the walkthrough, evaluators (often usability experts) simulate a user’s thought process at each step of the task. Your focus should be on the following cognitive walkthroughs questions:
- Will the user try to achieve the right effect? Does the user know what to do at this step?
- Will the user notice that the correct action is available? Is it clear what actions can be taken next?
- Will the user associate the correct action with the effect that the user is trying to achieve? Does the interface communicate how actions lead to outcomes?
- If the correct action is performed, will the user see that progress is being made toward the solution of the task? Does the system provide feedback that helps the user understand that they are on the right track?
💡 Pro Tip
Step through the tasks: assume the role of a user and go through tasks one at a time. Follow the predefined sequence, thinking aloud about each step, decision, and interaction.
- Identify usability problems: at each step, identify where users might encounter difficulties due to the design of the interface. These issues could involve the visibility of controls, labeling and grouping of information, feedback provided by the system, or assumptions about the user’s knowledge.
- Document findings: record and document identified usability issues, including task descriptions, user personas, and specific interactions. Include screenshots or videos where necessary to provide visual context.
- Recommend improvements: suggest specific design changes, clarifications, or enhancements to address usability problems.
- Collaborate for insights: cognitive walkthroughs are most effective as a collaborative process. Involving multiple evaluators or stakeholders can provide diverse perspectives and valuable insights.
Watch our video to learn more about the cognitive walkthrough and how to conduct it:
The cognitive walkthrough method
Cognitive walkthroughs are a formal method with a defined structure, specifying roles, session flow, and key questions to be discussed. This tried-and-tested approach yields reliable results. As you gain experience, you can adapt the format to suit your workflow, but be careful to maintain the core structure to ensure quality outcomes.
How are cognitive walkthroughs different from other usability tests?
At a glance, cognitive walkthroughs may appear similar to a prototype user test, or other usability tests. In some settings, they may, in fact, be very similar, but if you’re following the documented methods, there are a number of differences.
Cognitive walkthroughs | Other usability tests |
Focused on learnability | Can assess any aspect of the UX |
Audience is new users | Works for both new and experienced users |
Participants are internal, with set roles | External testing pool, reflecting real users |
Small testing group | Can include the widest possible group |
Suited to low-fi mockups | Often offers a working prototype |
When to use cognitive walkthroughs in product development?
Cognitive walkthroughs provide insights into any aspect of a product that requires users to perform a new task.
They are especially effective during conceptual work phases, as the outcomes can be high-impact, even when the prototypes are rudimentary.
Since cognitive walkthroughs are easy to set up and administer, the technique can be used throughout product design and development, even on deployed products, at a relatively high frequency.
Typically, the technique is used when:
- The product, feature, or screen requires users to learn how to use it
- The UX addresses a problem in a way that is different to existing norms
- The user is required to complete complex workflows, or
- It is likely that the user is unfamiliar or completely new to this kind of flow
💡 Pro Tip
Be aware that the outcomes are shaped by the testing group, which is typically a limited, internal set.
That’s why it’s important to follow up the cognitive walkthrough with user testing later in the product design to see if it works well for real users.
Cognitive walkthrough participant group options
In principle, a Cognitive Walkthrough is done by a small group of people who work in the product organization. This may be designers, product managers, engineers, IAs, department leads, in fact, anyone who may have a relevant perspective. Sticking with this group should make walkthroughs simple to organize and true to their original form and benefits.
However, it is also possible to conduct a cognitive walkthrough with larger groups, as well as with testers from outside the product organization. As long as the cognitive walkthrough participant, question, and method structure are maintained, you should still gain excellent qualitative and quantitative results.
This approach takes longer to organize and conduct, and is unlikely to be conducted quickly or often. Moreover, it’s important to source a pool of relevant users, which in the case of cognitive walkthrough means that they are new to the media being tested and are in the product target audience.
Where did cognitive walkthrough come from?
For as long as people have been inventing new systems, they have tested how efficient those systems are for new users. Early software engineers created processes to review and test the sequence of actions that users need to take in order to interact with their projects, which is the origin of cognitive walkthrough for product development.
In the 1990s, a group of researchers created a walkthrough methodology initially in reference to assessing ‘walk-up-and-use’ interfaces, such as ATMs. During the 90s, they evolved the method to be directly orientated to websites and coined the term ‘Cognitive Walkthrough’. Today, that same format is used in the evaluation of all kinds of digital products, with different practitioners adding their own refinements and variants.
Heuristic testing compared to cognitive walkthrough
- Heuristic evaluation (Nielsen and Molich, 1990; Nielsen, 1994) tests how well an interface performs against a set of principles that form the foundation of good user experience.
- Cognitive walkthroughs focus on evaluating new features, systems, and approaches, with a different structure from heuristic evaluation.
Heuristic evaluation provides feedback on how easy your product is to use, while cognitive walkthroughs are a formal review of how easy it is to learn.
The participants in a cognitive walkthrough
In its original form, a cognitive walkthrough comprises participants who represent the broader product team.
This includes:
- Product experts who know all about this product
- Domain experts who know all about this market
- One or more UX practitioners
- One or more Engineers
In the modern context, there is also room for product managers, product owners, UI designers, UX writers, information architects, UX researchers, and anyone else on a product squad. Certain organizations may also involve senior managers, founders, and even those in sales and marketing.
And, in actual practice, the participants are often only a group of product or UX designers!
The four questions always used in cognitive walkthroughs
The fundamental characteristic of a cognitive walkthrough is the questions which are to be discussed. Their original wording is still useful and relevant; however, there is a simplified and more universal version from Chris Kimmer which we prefer.
You can use either, or you may find further options for all or any of the four questions that suit you better.
Cognitive Walkthrough original question | …in simple terms | |
1 | Will users be trying to produce whatever effect the action has? | Will the user be trying to achieve the right effect? |
2 | Will users see the control (button, menu, switch, etc.) for the action? | Is the action visible? |
3 | Once users find the control, will they recognize that it produces the effect they want? | Will the user recognize the action as the correct one? |
4 | After the action is taken, will users understand the feedback they receive so that they can go on to the next action with confidence? | Will the user understand the feedback? |
Cognitive walkthrough planning
A small amount of planning for your cognitive walkthrough will help the session run well and maximize your chances of meaningful results. A good cognitive walkthrough agenda will cover the following information.
1. Overview
- Purpose – The purpose of the workshop, what is being evaluated, and why?
- Audience – Who are the personas that the evaluators should empathize with?
- Requirements – List the prototypes, tools and spaces to be used
- Rigor – Are there any challenges to research rigor and objectivity, and how will they be overcome?
- Outcomes – Outline what artifacts will be created and their use.
2. People
List the specific people required to do the workshop, and what role each will play. It is acceptable for the same person to perform multiple roles. A cognitive walkthrough requires the following roles to be performed.
- Facilitator – Runs the session.
- Presenter – Operates the prototype.
- Recorder – Takes notes and collates them after the session.
- Evaluators – Discuss the four questions.
3. Subjects
Make a list or table of each sequence or goal to be evaluated. Note if there are any aspects specific to each sequence, such as a different persona, scenario, or prototype.
4. Preparing a cognitive walkthrough prototype
Although we’re only touching on it lightly here, preparing the prototype or prototypes for a walkthrough workshop should be the main preparation activity. It may not be a painstaking task though; remember, these prototypes only need to be as finished as is necessary to demonstrate the concept.
Once you have completed the above agenda and your prototype, you’re ready to conduct the workshop. Typically, the agenda is circulated either before or at the beginning of the session, however, be mindful of the potential of this kind of information to influence the results. You should always strive to create an agenda, but if you feel it will skew the workshop outcomes, there is no need to circulate it.
How to conduct a cognitive walkthrough workshop
The final aspect of the cognitive walkthrough method is the workshop itself. The following sequence is recommended for a classic ‘internal’ workshop.
- The Facilitator starts by talking through the goals, roles, personas, rules, and optionally, the agenda.
- The Presenter steps through the interface, stopping at each decision point.
- The Evaluators discuss each of the four questions and agree to an action at each step.
- The Presenter takes the agreed actions.
- The Recorder notes success or failure, and any qualitative feedback.
- The Recorder summarizes and circulates the outcomes after the workshop.
Ideally, your workshop will run closely to this format, but in practice, you should expect it to vary a lot from the format. The outcomes are still valid, and more so if you’ve been able to keep to the four questions.
Workshops with external participants will need to be adapted from this format to suit the specific setting.
Tips for conducting a Cognitive Walkthrough workshop:
- Overemphasize personas to the evaluators – it’s important that they’re empathetic with your actual new users.
- Keep laptops closed – we need full attention!
- Do not try to justify the prototype – participants should talk about what they encounter, not why it is the way it is, with strong focus on user reaction.
- When testing with internal participants it can be helpful to really limit the number of participants.
- Conversely, a larger group of external participants can create more meaningful results.
- Test microcopy options, not just button placement.
- For an external walkthrough, consider using a mixed-method approach, such as a survey or interview to add depth to observations.
Conclusion: The benefits of cognitive walkthroughs in collaborative work
Incorporating a cognitive walkthrough structure into the regular practice of stepping back and discussing your work with colleagues adds rigor and enhances the potential to improve quality at a relatively low cost. From their origins in early software programming to their evolution for walk-up-and-use systems, and their current application in evaluating digital products, walkthroughs remain a valuable and enduring tool.
To streamline your research and gain better insights into user behavior, sign up for a free UXtweak account and explore our range of testing tools!
Sources:
[Clayton Lewis, Peter Polson, Cathleen Wharton, John Reiman, Testing a Walkthrough Methodology for Theory-Based Design of Walk-Up-and-Use Interfaces, 1990 https://www.cs.helsinki.fi/u/thusu/opinnot/kaytarv/artikkelit/p235-lewis.pdf ]
[Clayton Lewis, Peter Polson, Cathleen Wharton, John Reiman, The cognitive walkthrough: A practitioner’s guide, 1994 https://www.colorado.edu/ics/sites/default/files/attached-files/93-07.pdf ]
