HUMAN-COMPUTER INTERACTION SECOND EDITION
Dix, Finlay, Abowd and Beale

Search Results

Search results for evaluation
Showing 61 to 70 of 110 [<< prev] [next >>] [new search]

Chapter 11 Evaluation techniques 11.4.4 Model-based evaluation Page 415

The final approach to evaluating the design that we will note is the use of models. Certain cognitive and design models provide a means of combining design specification and evaluation into the same framework. For example, the GOMS model (see Chapter 6) predicts user performance with a particular interface and can be used to filter particular design options. Similarly, lower-level modelling techniques such as the keystroke-level model (Chapter 6) provide predictions of the time users will take to perform low-level physical tasks.

Chapter 11 Evaluation techniques 11.4.4 Model-based evaluation Page 415

Design methodologies, such as design rationale (see Chapter 5), also have a role to play in evaluation at the design stage. Design rationale provides a framework in which design options can be evaluated. By examining the criteria that are associated with each option in the design, and the evidence that is provided to support these criteria, informed judgements can be made in the design.

Chapter 11 Evaluation techniques 11.5 Evaluating the implementation Page 415

The techniques we have considered so far concentrate on evaluating the design before it exists as a runnable system, although as we have noted they can also be used with implementations. They typically involve analysis by the designer (or an expert evaluator) rather than testing with actual users. However, vital as these techniques are for filtering and refining the design, they are not a replacement for actual usability testing with the people for whom the system is intended: the users. In this section we will look at a number of different approaches to user-centred evaluation. These include empirical or experimental methods, observational methods and techniques which we will call query techniques, which ask the user for feedback directly.

Chapter 11 Evaluation techniques 11.5 Evaluating the implementation Page 416

The major difference between this type of evaluation and evaluation of design, aside from the involvement of the user, is the existence of an actual implementation of the system in some form. This may range from a simulation of the system's interactive capabilities, without its underlying functionality (for example, the Wizard of Oz technique, which is discussed in Chapter 5), through a basic functional prototype to a fully implemented system. The evaluation techniques described in this section can be used to evaluate any of these.

Chapter 11 Evaluation techniques 11.5.1 Empirical methods: experimental evaluation Page 416

11.5.1 Empirical methods: experimental evaluation

Chapter 11 Evaluation techniques Subjects Page 416

The choice of subjects is vital to the success of any experiment. In evaluation experiments subjects should be chosen to match the expected user population as closely as possible. Ideally this will involve experimental testing with the actual users but this is not always possible. If subjects are not actual users they should be chosen to be of a similar age and level of education as the intended user group. Their experience with computers in general, and with systems related to that being tested, should be similar, as should their experience or knowledge of the task domain. It is no good testing an interface designed to be used by the general public on a subject set made up of computer science undergraduates: they are simply not representative of the intended user population.

Chapter 11 Evaluation techniques Variables Page 417

Independent variables are those characteristics of the experiment which are manipulated to produce different conditions for comparison. Examples of independent variables in evaluation experiments are interface style, level of help, number of menu items and icon design. Each of these variables can be given a number of different values; each value that is used in an experiment is known as a level of the variable. So, for example, an experiment that wants to test whether search speed improves as the number of menu items decreases may consider menus with five, seven, and 10 items. Here the independent variable, number of menu items, has three levels.

Chapter 11 Evaluation techniques Variables Page 417

Dependent variables on the other hand are the variables which can be measured in the experiment. In the example given above, this would be the speed of menu selection. The dependent variable must be measurable in some way, it must be affected by the independent variable, and, as far as possible, unaffected by other factors. Common choices of dependent variable in evaluation experiments are the time taken to complete a task, the number of errors made, user preference and the quality of the user's performance. Obviously some of these are easier to measure objectively than others. However, the more subjective measures can be applied against predetermined scales, and can be very important factors to consider.

Chapter 11 Evaluation techniques Think aloud and cooperative evaluation Page 427

Think aloud and cooperative evaluation

Chapter 11 Evaluation techniques Think aloud and cooperative evaluation Page 427

Think aloud has the advantage of simplicity; it requires little expertise to perform and can provide useful insight into problems with an interface. Also it may be used to observe how the system is actually used. It can be used for evaluation throughout the design process using paper or simulated mock-ups for the earlier stages. However, the information provided is often subjective and may be selective, depending on the tasks provided. The process of observation can alter the way that people perform tasks and so provide a biased view. The very act of describing what you are doing often changes the way you do it -- like the joke about the centipede who was asked how he walked Š

processed in 0.004 seconds