Human-Computer Interaction 3e Dix, Finlay, Abowd, Beale
exercises - 9. evaluation techniques
EXERCISE 9.3
In Section 9.4.2 (An example: evaluating
icon designs), we saw that the observed results
could be the result of interference. Can you think
of alternative designs that may make this less likely?
Remember that individual variation was very high,
so you must retain a within-subjects design, but you
may perform more tests on each participant.
answer
Three possible ways of reducing interference
are:
- During the initial training period,
swap back and forth between learning the two sets
of icons, with the aim of getting the subjects used
to swapping between the two sets of remembered icons.
However, this design could be argued to suffer the
same flaws as the original. If the abstract icons
had been taught in isolation perhaps they might
have fared far better.
- We could invent a third set of 'random'
icons (call them R). We could then interpose them
in the experiment, that is present the icons in
the orders RARN and RNRA. The intention is to swamp
any transfer effect in the 'noise' of the random
icons. It could be argued that our experiment then
measures the robustness of the icon sets to such
'noise'!
- We could give the subjects multiple
presentations, for example ANAN and NANA presentation
orders. This would not remove transfer effects,
but it would give us some way to quantify them.
Imagine that in the ANAN group the second presentation
of the abstract icons was significantly worse than
the first, but there was not a similar effect for
natural icons in the NANA group. This would give
us both positive evidence of a transfer effect,
and perhaps some quantitative measure. However,
even going from this additional evidence to a strong
conclusion will be difficult.
Notice that all the above measures require
additional subject time and one has to constantly
weigh up the advantages of richer experiments against
those of larger subject groups.
Other exercises in this chapter
ex.9.1 (ans), ex.9.2 (ans), ex.9.3 (ans), ex.9.4 (ans), ex.9.5 (ans), ex.9.6 (tut), ex.9.7 (tut)
all exercises for this chapter