HUMAN-COMPUTER INTERACTION SECOND EDITION
Dix, Finlay, Abowd and Beale

Search Results

Search results for evaluation
Showing 91 to 100 of 110 [<< prev] [next >>] [new search]

Chapter 11 Evaluation techniques Immediacy of response Page 438

Another factor distinguishing evaluation techniques is the immediacy of the response they provide. Some methods, such as think aloud, record the user's behaviour at the time of the interaction itself. Others, such as post-task walkthrough, rely on the user's recollection of events. Such recollection is liable to suffer from bias in recall and reconstruction, with users interpreting events according to their preconceptions. Recall may also be incomplete. However, immediate techniques can also be problematic, since the process of measurement can actually alter the way the user works.

Chapter 11 Evaluation techniques Intrusiveness Page 438

Related to the immediacy of the response is the intrusiveness of the technique itself. Certain techniques, particularly those which produce immediate measurements, are obvious to the user during the interaction and therefore run the risk of influencing the way the user behaves. Sensitive activity on the part of the evaluator can help to reduce this but cannot remove it altogether. Most immediate evaluation techniques are intrusive, with the exception of automatic system logging. Unfortunately this is limited in the information that it can provide and is difficult to interpret.

Chapter 11 Evaluation techniques Resources Page 438

The final consideration when selecting an evaluation technique is the availability of resources. Resources to consider include equipment, time, money, subjects, expertise of evaluator and context. Some decisions are forced by resource limitations: it is not possible to produce a video protocol without access to a video camera (and probably editing facilities to boot). However, other decisions are not so clear cut. For example, time and money may be limited forcing a choice between two possible evaluations. In these circumstances, the evaluator must decide which evaluation tactic will produce the most effective and useful information for the system under consideration. It may be possible to use results from other people's experiments to avoid having to conduct new experiments.

Chapter 11 Evaluation techniques Resources Page 438

Finally the context in which evaluation can occur will influence what can be done. For practical reasons it may not be possible to gain access to the intended users of a system (if it is a general system, for example) or it may not be feasible to test the system in its intended environment (for example, a system for a space station or a defence system). In these circumstances simulations must be used.

Chapter 11 Evaluation techniques 11.6.2 A classification of evaluation techniques Page 439

11.6.2 A classification of evaluation techniques

Chapter 11 Evaluation techniques 11.6.2 A classification of evaluation techniques Page 439

Using the factors discussed in the previous section we can classify the evaluation techniques we have considered in this chapter. This allows us to identify the techniques which most closely fit our requirements. Table 11.4 shows the classification for analytic techniques, Table 11.5 for experimental and query techniques and Table 11.6 for observational techniques.

Chapter 11 Evaluation techniques 11.7 Summary Page 440

Evaluation is an integral part of the design process and should take place throughout the design life cycle. Its aim is to test the functionality and usability of the design and to identify and rectify any problems. It can take place in the laboratory or in the user's workplace, and may involve active participation on the part of the user.

Chapter 11 Evaluation techniques 11.7 Summary Page 440

A design can be evaluated before any implementation work has started, to minimize the cost of early design errors. Most techniques for evaluation at this stage are analytic and involve using an expert to assess the design against cognitive and usability principles. Previous experimental results and modelling approaches can also provide insight at this stage. Once an artefact has been developed (whether a prototype or full system), experimental and observational techniques can be used to get both quantitative and qualitative results. Query techniques provide subjective information from the user.

Chapter 11 Evaluation techniques 11.7 Summary Page 440

The choice of evaluation method is largely dependent on what is required of the evaluation. Evaluation methods vary in the stage at which they are commonly used and where they can be used. Some are more subjective than others and provide qualitative rather than quantitative measures. Some provide immediate information while others get feedback after the event. However, the more immediate methods also tend to intrude most seriously on the interaction. Finally some require more resources in terms of time, equipment and expertise than others.

Chapter 11 Evaluation techniques Exercises Page 441

11.4 Choose an appropriate evaluation method for each of the following situations. In each case identify

processed in 0.003 seconds