Wednesday, August 22, 2018

Expert Reviews (Heuristic Reviews)

Expert reviews (also called heuristic reviews) can identify most of the usability issues that are likely to occur.  In the hands of a competent designer, this type of review can be extremely effective. We have validated the quality of our expert reviews by conducting empirical usability tests on the pages we’ve reviewed. In general we find that careful and thorough reviews catch most of the problems with only a few additional problems being encountered in the usability test. Because expert reviews are less expensive than usability tests, we like to use them frequently.

Expert - or heuristic - reviews are an opportunity for the design team to obtain the perspectives of designers who have been outside the development process. There is probably no single more cost-effective way to improve the quality of a product.

Why Expert Reviews are Valuable

An outside perspective can be extremely valuable. When a software product is released, one of the most important questions is "will users find it intuitive"? Although the design team has struggled with this question throughout the entire project, they become so enmeshed in the process that they can no longer view it with naive eyes. They have been involved in many debates and compromises. In addition, the design team has confronted the realities of schedules, budgets, working with marketing and technical colleagues who may have different priorities and dealing with management expectations.

If we could predict the future, we could really reduce risk. Since we can't, the best we can do is to try to identify elements in the design which may be barriers to the users and impair the product's acceptance. The goal is to make the groups of successful users as large as possible and to have users feel that the software is a real asset - a tool that enhances their productivity.

Two tools that we have to help in this process are heuristic design reviews and usability tests. These two techniques complement each other. Usability tests show us how new users relate to the product on initial exposure. Unfortunately, usability tests are also expensive and time-consuming. And their yield can be relatively low compared to the effort expended. Usability testing is most useful at the beginning of a project, when we want to see if users "get" the basic design concept, and at the end, to identify any low-effort, high-yield refinements which are still possible.

A heuristic review is a consultative process. While there are a number of different techniques which can be used, the core process is to ask experienced designers, to work through the software, adopting as far as possible the viewpoint of a novice user. This is perhaps analogous to a teacher asking another educator to look at some instructional material to assess how well students will work with it.

Cognetics' approach is to conduct a heuristic review with 2-3 designers. This is a large enough group to provide a diversity of opinion yet small enough to be efficient. Before coming together, each designer looks at the software alone. The first look at a product is very valuable. It is important for the designer to make notes of initial reactions, misconceptions and blunders before he or she becomes too familiar with the software. Having walked through the software for initial reactions, the designer re-visits the software. He or she now looks at each screen looking for violations of the Cognetics heuristic guidelines. They note issues such as:

  1. What problems, if any, did I have here? Why was I confused and how might that confusion have been reduced or avoided.
  2. What elements might confuse novice users, even though I was not confused?
  3. What concepts and skills must the user have to understand the material presented at this point and the options available? Are there ways in which the presentation and options might be simplified.
  4. Are there demands on the user's recall (both conceptual and short-term memory) which could be reduced.

At the conclusion of this process, the designer should have developed a comprehensive set of notes, have reasonably well-developed ideas about design issues and possible solutions.

Now the design team comes together and performs a second walk-though of the software. As they go through it together, they raise and discuss the issues they noted. Some issues may be dismissed while new ones may emerge. Solutions are also suggested. At the conclusion of this process (usually ended by exhaustion) the combined set of notes is organized and a report created.

The results of the review are usually presented in a work session with the software design team. It is important that heuristic reviews not be seen as attacks on the design but as collegial input. It is not our role to make judgements about the design (like a movie critic) but to provide input that can help a software team achieve its goals.

The Expert Review Process

Expert reviews are often done as the beginning of a redesign project, as a way of correcting design or usability flaws before a formal usability test, or as part of a design engagement to solve a problem which cannot be resolved from within the current team.

The details of the schedule will vary, but typically they are structured in three steps:

  • Cognetics reviews the software
    The Cognetics designers receive, install and review the software on their own, to approximate the new users’ experience, and record their comments and observations. For complex software or task domains, an expert will need to be available to the review team to answer questions about the software and its typical uses.
  • Cognetics assembles the notes from the review observations
    Review observations are recorded and compiled with a brief summary describing the areas where the interface needs the most improvement.
  • Cognetics works with the product team to solve usability problems found in the review
    The Cognetics designers then meet with the product team to discuss the observations and work on design solutions to solve the problem. This session lasts from one to two days. A final day is spent, when necessary, working with the development or product lead to assemble the notes from the working session.

The goal of this review structure is to shift the emphasis from discovering problems to solving them, with the active involvement of the product development team. The review becomes part of the design process, rather than a "test" of the team's capabilities.

Thanks for contacting us. We'll be in touch shortly. Charlie & Anne

 What Makes Web 2.0 and Social Computing So Difficult?

  There are Ten Barriers to Web 2.0:

  1. Fear of hostile or embarassing postings that can damage your image.
  2. Concern that staff may act inappropriately or disclose proprietary information.
  3. Cost and time required for content development and maintenence.
  4. Compliance, regulatory and legal issues.
  5. The Generation Gap: employees have different expectations and use technology differently.
  6. Requirement for deep competence in usability and user experience engineering.
  7. Cross-departmental, cross-functional, cross-hierarchical, unfiltered communications.
  8. Concerns about security.
  9. What you don't know about how the competition is using these tools to their advantage.
  10. Balancing control with transparency.

  Web 2.0 is defined by agility, transparency, empowerment, user-centricity and creativity... not words typically used to describe most organizations. Managers are not prepared to manage in this environment.


Picture of Charlie Kreitzberg

Charles B. Kreitzberg, Ph.D.

is founder and CEO of Cognetics Corporation, a company that, since 1982, has created award-winning interactive designs that connect people and teams with computers.

Dr. Kreitzberg has designed the user interfaces for websites, software and rich internet applications for clients all over the world.

He developed a pre-web browser, is author of LUCID – the a widely used interaction design framework, and most recently In the Know!™ -- a software knowledge and communications tool for teams and their leaders.

Dr. Kreitzberg has lectured and consulted at corporations and universities worldwide. He has served as an expert witness in software interface patent disputes. He is Founding Editor of User Experience magazine; he has authored numerous articles and has served on the national boards of the Usability Professionals Association and the Society for Information Management. He holds advanced degrees in computer science and psychology. Dr. Kreitzberg serves as the Technology Director for Einstein’s Alley.

Download Charlie's Detailed Vita

Vista users -- if you are using an older version of Adobe Reader, you may have a problem with the PDF. The solution is to upgrade your version to version 9 (or better). This is an Adobe problem and we appologize for the inconvenience.

Download The First 10 Seconds as a PDF

Our Workshops

Our workshops are customized for you.people gathered around a laptop computer

Here are some sample agendas:

Higher Education




Business Leaders

Cancel   Send
Please share your thoughts with us. Please note that comments will be reviewed before posting to avoid spam.
Average rating:  (0)


Comment *
Enter code!

Privacy Statement  |  Terms Of Use
Copyright 2007-2015 by Cognetics Corporation