A heuristic evaluation case.
We can define comparative historical research as a homogeneous set of techniques that facilitate comparative knowledge of national or international realities. With this type of research a broader and deeper knowledge of the subject matter is achieved. With the comparative methodology we will be able to understand other realities in their own contexts, comparing them with each other and with our own reality, and draw operational conclusions.
A priori, it may seem that such an academic type of research has no place in the business environment of a product team. But, we are going to analyze each of the steps that we must follow in an investigation like this. Then, we are going to illustrate it with a real example that we have carried out in the Liferay Design department.
The first step that every researcher must take is the choice of the topic and its definition in space time.
For the choice of the topic, three criteria are usually discussed: relevance criterion, viability criterion and originality criterion.
Our chosen topic was heuristic evaluation as a method of analysis in design.
We consider that it is relevant because by studying the past of the heuristics we solve present problems regarding their use in our team.
Regarding viability, we had to take into account that we did not have much time to carry out the research. This impacted certain decisions like the selection of the studied documentation.
Regarding the criterion of originality, despite being a rather hackneyed subject, for internal reasons of the design department, we wanted to make our own approach to the subject based on review of existing interpretations.
In this next phase, reading the bibliography is essential. This allows us to delimit the topic based on what is already investigated.
Exhaustive research on heuristic evaluation from its inception would have taken years of dedication, time we did not have. Therefore, we made an incipient approach to the most relevant documentation to which we had access. We reviewed the contributions of John D. Gould and Clayton Lewis (1985), Ben Shneiderman (1992), Christian Bastien & Dominique Scapin (1992), Jakob Nielsen (1994), Gerhardt-Powals (1996), Weinschenk and Barker (2000), Connell & Hammon (2000), and Bruce Tognazzini’s (2003). Then we compiled a list of almost 30 documents including articles, conferences and doctoral theses of the most recent approaches to the subject (Granollers (2018), Suárez (2011); Paz (2018), Murillo (2018), Quiñones (2018); Kurosu (2018); Cooper (2007)).
In this phase, it is best to summarize the most relevant findings from each document using cards. These cards can then be classified and ordered in a clear and simple way. They should allow a quick consultation. It is advisable that these cards contain both source information and content.
Cards can be made on paper or digital. We decided to work with Notion. Being a collaborative investigation between three people, we consider it to be a way of making information more accessible to everyone.
Finally, once all the information was analyzed, we discovered an evolution in the approach to the topic from the past to the present. Traditional heuristic principles propose a more general evaluation. On the other hand, the most modern authors seek a more concrete definition of the evaluation criteria, break them down into easy-to-identify sub-principles and try to quantify, as much as possible, the evaluation results.
We did not want to stay in the comparison phase. We wanted to approach the creation of a new heuristic evaluation model that would adjust to the needs of our product teams.
We used the card sorting method to group all the heuristic sub-principles that we had detected in the analysis stage into groups with common thematic characteristics. We reviewed each one, justifying its inclusion through references to different authors who talked about the subject.
After three iterations, we developed a heuristic evaluation list comprising 11 heuristic principles, with a total of 57 sub-principles.
We all know that everything happens very quickly in a product team and sometimes think doing such an exhaustive analysis of a topic is a waste of time. However, comparative research allows us to gain a deep knowledge of a topic, and only from this point can we start to grow. It is not a waste of time but an investment in knowledge that is later applied with practical tools such as a heuristic analysis table.
Patience, tenacity, dedication, and thoroughness are essential attitudes when undertaking an investigation of this type that, as you have observed, bears wonderful results.