Ego, Emotion and Experience

Draft of Chapter for next edition of Dix, Finlay, Abowd Beale, Human–Computer Interaction, Prentice Hall

This version still includes authors' notes, and some parts are incomplete.

Ego, Emotion and Experience / Aesthetics, Persuasion and Motivation

Which chapter title do you like best?

S1.1 Introduction

Whilst older images of computation have typically been clinical and emotionless, increasingly computer systems are part of leisure activities, home life and fun. There is a long history of computer games, but increasingly artwork is also incorporating digital elements and of course consumer products such as the iPod not only perform functions, but also enchant and captivate their users.

Emotion is a basic part of our make-up as humans, not an optional extra, but an essential part of who we are. So our designs for computer systems and digital products need to take account of the joys, fears, excitement, trust and insecurities, not just our speed of typing.

Emotion may be a primary or secondary goal of a product:

primary goal

eliciting emotion is the purpose of the product as is the case of a game, artwork, or entertainment system. The functional aspects of such systems (e.g. selecting a TV channel) are there to support the ultimate emotional goal.

secondary goal

the primary goal of the system is some other functional purpose, but appropriate emotions help the user to perform the task well or efficiency. For example, in a repetitive job we want the system to engage interest and hence maintain alertness, or in an educational system we want to increase the student's motivation and hence rate of learning.

These two classes are of course not completely disjoint. For example, the purpose of an online shopping site is to allow customers to buy products. However, shopping is more than buying, it is an experience and the enjoyment of using the site and choosing what to buy may be as important as the purchase itself. Furthermore, the level of trust the site develops in the customer may make the difference between actually paying money or simply browsing then going elsewhere.

When considering both the human and the computer in Chapters 1 and 2, we considered input and output between user and system. In a similar way we can consider the 'direction' of emotion between system and user. Systems may:

elicit emotion

The system is be designed in order to create a particular feeling in the user. For example, good graphic design might create a feeling of dependability for a bank, or excitement in a web site for a holiday resort.

detect and respond to emotion

The system uses various means to detect the user's current emotions. Techniques include sensing physiological indicators such as heart rate, detecting the kind of language used in messages, or using cameras to recognise facial expressions. This knowledge of the user's emotions can then be used to modify and adapt the system's responses to the user. For example, if the user appears tired a car audio system might change the genre of music, if a game player appears bored the game might increase the difficulty.

act as conduit for emotion

The system in some way records or transmits things which enable users to express emotions to one another or to understand the emotions of others. For example, emoticons in instant messenger allow you to change the way a message is understood.

S1.2 Theories of emotion, motivation and experience

S1.2.1 Psychology, physiology and sociality of emotion

Although there was interest in the psychology of emotions towards the end of the 19^th and early 20^th century there was a long gap in studying emotions throughout most of the mid to late 20^th century largely because it could not easily be numerically measured and the need to take subjective experience into account went against the scientific fashions of the era. Happily there is now an increasing amount of work in the area – helped not least by the fact that more is measurable!

Kinds of emotion

Psychologists usually distinguish emotion from feeling. In day-to-day speech the two are largely synonymous, but those formally studying the area use emotion to mean the physiological and neurological state of the body whereas feeling refers to the subjective experience. They also distinguish mood which are longer term positive or negative feelings, in this case similar to the common use of the word.

These three are not separate things but interlinked. You hear a loud bang and your body instinctively exhibits a response to the emotion of fear (measurable through skin resistance, heart rate etc.) and this gives rise to a feeling of shock. However you immediately realise this is due to a friend bursting a balloon and your feeling of shock turns to amusement, this feeling then changes your emotion and as you laugh your body shakes with more positive emotion. However, if your overall mood had been negative, perhaps you had a bad day, the fear–relief feelings might instead lead to annoyance and instead of laughing you might shout at your friend.

Psychologists also typically distinguish more basic emotions such as fear, anger and happiness, which we share with many animals, from more complex ones such as shame, anxiety, and regret. The more basic emotions are associated with the part of the brain known as the limbic system, which is similar to the brain of reptiles and some emotional responses (such as the loud sound to fear response) are triggered directly by this. However, as in the vignette above, higher levels of your brain instantly interpret the emotions in the lower brain and make sense of them in a broader context.

When you are very frightened or very happy you feel your heart thumping in your chest. In addition your face will go red as the blood vessels dilate, your skin resistance will drop as your body is coated with a fine layer of sweat and inside various chemicals wash through your blood stream. This state is called arousal and is common to positive and negative emotions. The direction of the emotion, positive or negative is called its valance. Arousal is important in physiological computing partly because it is relatively easy to measure, but interpreting emotion from arousal, even deciding its valance, is far harder.

Not just about the head.

Emotion is not just about what goes on in your brain, nor even the chemicals flooding inside, but influences and is influenced by your whole body and the world around.

When the balloon pops and your lower brain registers its instinctive fear response, this triggers changes throughout your body. Your heart rate goes up and adrenalin is released into your blood stream. This is known as the 'flight or fight' reaction and the adrenalin raises all your bodily functions ready to make some action. When you realise it is the balloon and start to laugh the tension in your body that was ready to run away and attack is instead released in the intense muscular movements of the laughter. In addition as you laugh neurochemicals are released in your brain including ones very like opiates.

The physical movements of your body also feedback into this electrical-chemical story. That a brisk walk is not just good for your health it really does make you 'feel' better! The physical exercise is working on your bodies chemical system, but also there is direct feedback to your brain. In one set of experiments subjects are asked to manipulate their face muscle by muscle and then hold the expression [SM88]. Because they have no mirror they do not know that their face is being set into expressions of happiness, sadness, anger and joy – yet even though they cannot see themselves and cannot consciously recognise the expressions when tested they actually feel the emotions their faces are set to. It appears that the same sort of interpretation systems that makes your brain say "I feel fear why is that?" also works for the smile "my face is smiling I must be happy". As the song say "put on a happy face".

Finally our emotions are influenced by our environment, not least the people around us. In another experiment psychologists created a series of cocktail parties where the subjects believed this was prior to the experiment. An experimenter went into the party and acted either friendly or aggressively for different groups. The aim was to see how the subjects responded, but an unexpected effect was that the experimenters who went in acting happy came out feeling happy, the ones who went in acting aggressively came out feeling aggressive . This will be partly due to the 'happy face' effect – their brains say "I'm acting happy, I must be happy". However, in addition, if you smile at people they smile back, so you feel happy because they smile at you. However, your interpretation of other people's actions and the state of the world also depends on your cultural attitudes, personality and current emotional state. So the psychologist who smiles at you might make you feel happy, or suspicious.

These multiple positive feedback loops between brain activity, body and brain chemicals, bodily movement and social interactions are why individuals and even more groups can swing quite suddenly into extreme emotional situations.

S1.2.2 Reason and emotion

In Star Trek the Vulcan race are ruled by logic and feel no need for emotion. This reflects a popular view that reason and emotion are in some way opposed. This view is also central to much of traditional philosophy and until recently most psychological science. In recent years however it has become evident that emotion and reason are in fact much more closely interlinked than had been thought. Instead of emotion being seen as something that distracts us from proper reasoned logical thought, instead it is increasingly been seen as a way of guiding us efficiently to the correct decisions and being essential to many apparently purely logical problem solving tasks.

One of the key clues to the importance of emotion in reasoning and decision making has been studies of people with specific brain injuries or lesions which have dulled one aspect or other of their emotional system. Antonio Damasio's books have been central in communicating and popularising the importance of emotion and describe several patients with such injuries [Da00]. One patient, Elliot, had suffered damage to his prefrontal cortex due to a 'benign' brain tumour. On all tests of reasoning, both logical and ethical, he functioned normally, but yet could not make decisions. In addition, Elliot seemed emotionally cold, able to see the impact of things, but apparently not able to feel it.

To see why emotion is important to decision making consider the paradox of Buridan's Ass, named after a 13^th century philosopher, but originally proposed by Aristotle. Imagine placing an ass (or donkey) exactly half way between two equally good bales of hay. The donkey needs to choose which one to eat, but for the totally rational donkey there is no way to choose one rather than the other ... and so the poor donkey would starve. Happily real donkeys are a lot more sensible than purely rational philosophers and would go one way or other for some totally irrational reason. To resolve just this kind of dilemma, many computer algorithms use random choices and such non-deterministic or probabilistic algorithms often far outperform their purely deterministic counterparts .

However, emotion is not just a random influence. In many situations there are detectable differences between options, but we may be faced with choices where the criteria are incomparable or the outcomes uncertain. On a rainy day do I risk running across the road to catch my bus? If I do I will get home dry, but risk being hit by a car. Even if one knows all the probabilities, how do you weigh up a 1 in 10,000 risk of being run over with a 100% chance of getting wet? Mathematical decision procedures and computer algorithms often need complete knowledge in order to make fully 'rational' choices and where only partial knowledge is available they use heuristics, approximations, and crude hacks ... and end up behaving not so unlike emotional people.

Emotion also is important in maintaining focus and attention, the planning and control parts of thinking. Elliot eventually lost his job partly because he could get distracted very easily, or alternatively spend a whole day (a bit like Buridan's Ass) trying to decide on some insignificant decision. The idea of the bigger picture, that the whole is more important than a part was lost to him. Even simple boredom is important here, without boredom we will constantly try and retry very similar solutions rather than look for 'something completely different'.

Finally, as noted previous emotion is efficient! It is often better to make a poor decision quickly than make a good one over a long time. Yunga the cavewoman hears a rustling in the bushes. "It could be the wind", she thinks, "or a bird". Some time later she decides, on balance of probabilities it is most likely a sabre tooth tiger – oops too late!

Poltical cartoon inspired by Buridan's ass

S1.2.3 Maslow's Hierarchy of Needs

In 1943 Abraham Maslow described five basic levels of needs [Ma43,Ma70].

1.  'physiological' needs (physical)

These include hunger, thirst, and freedom from disease and harm. These are very basic drives of the body such as hunger and thirst trying to maintain homeostasis. Even at this level there are differences between extreme starvation needs and more day-to-day hungers.

2.  safety needs (security)

These include shelter and safety from harm. Maslow includes pain from illness at this level, although often other writers include this at level 1.

3.  love needs (social)

These include the desire to be loved and cared for by others, and the desire for friendship. At this level a sense of territory, tribe and class also emerge as important needs – the need to belong can also include the desire to exclude.

4.  esteem needs (ego)

These are needs related to feelings of self-worth, self-confidence and worthiness. This is played out partly in ones own eyes, but also significantly in those of others wishing approval, prestige and status.

5.  need for self-actualization

At this level we have self-fulfilment, the desire to be creative, to be oneself, to grow intellectually for its own sake

Maslow argued that it is only as the needs at lower levels are satisfied that those of the higher levels emerge and so these are often drawn as a pyramid with self-actualisation at the top (Figure MMM.). Clearly this is not entirely true on a moment-by-moment basis, for example a bungee jumper risks safety in order to satisfy a need for self-esteem or perhaps even self-actualisation. However, if you look at the development of civilisation you seen a general pattern towards consideration of the 'higher' needs, and furthermore if lower needs are not met for prolonged periods the natural (but not universal or uncontrollable) tendency is to shift ones focus increasingly towards them. Furthermore if you look at the progression of the uses of the computer, there is a tendency to see more recent innovations towards the higher levels in Maslow's hierarchy!

Maslow also wrote about cognitive needs, but did not place them in this hierarchy. In his 1943 paper he discussed "the desires to know and to understand" (why you are reading this book!), then in a later book where he revisited his ideas he talks about "aesthetic needs" appreciating and creating beauty. Some writers place these 'above' level 5 as yet higher level needs, but Maslow himself appears to regard them alongside. At these levels it seems that personality differences are far more significant and it is likely that the relative importance of these will differ from person to person and from time to time.

Many activities cross these levels, for example, a car may be sold on its safety features (level 2) or as a status symbol (level 4). Indeed it can be argued that one of the dangers of consumer society is that, encouraged by advertising, we gather material things, that really should be about level 1 and 2 needs, in order to satisfy unfulfilled needs for love, self-esteem and personal fulfilment. Note too that whilst influential, this schema was developed in a time when psychology was dominated by largely behavioural modes of thinking, so the hierarchy focuses on satisfaction of individual needs and so is more limited in its treatment of drives such as duty, giving or self-sacrifice.

Fig MMM. Maslow's hierarchy of needs

S1.2.4 Taking Play Seriously

It is interesting to note that Maslow's hierarchy does not have a place for play! Play seems the ultimate in superfluous activity, so in some senses it should be placed high in the hierarchy – something to do only if all lower needs are satisfied; yet hard to see having the same depth or import as self-actualisation. However, just as emotion is recognised as a critical part of reasoning, play has become a serious study for many researchers and increasingly we understand its importance in many aspects of personal development and corporate life.

Perhaps most obvious is the use of play in education, indeed some explanations of children's play centre on its educational value. Even a tiny baby hitting at hanging coloured baubles is discovering about the world and its own body; and in the schoolyard children experiment and grow to understand social relationships (including some of their aberrations).

Sometimes the play is an intrinsic part of an educational process, for example, playing football helps you to learn coordination as part of what it is. However, often educational software uses cruder means where the child has to do small puzzles in order to continue which seem dissociated to the game itself. The former is an example of internal motivation learning the skill carries its own reward, whereas the latter is and example of external motivation where the reward for doing the puzzle is to continue the game (the motivation).

This difference between internal and external motivation occurs not just in play and education, but in many aspects of life. For example, one may do a job because you enjoy it, or because you want the pay at the end of the week – the former is internal motivation the latter external motivation. It is usually assumed that internal motivation, often focused on Maslow's higher-level needs, is better than external motivation based on rewards and punishments.

Some theorists see play as an essential part of much of organised civilisation. As early as the 1930s, Johan Huizinga in his book Homo Ludens [Hu44] examined play throughout history and to modern times seeing elements of play in areas such as warfare, government and courts as well as more obvious places such as theatre and organised sport. While the exact nature of these activities differ between cultures and through time, there are common features:

boundaries

The play is bounded in time and space, you know when you physically or metaphorically step into the area of play

rules

The are agreed rules. The rules may be (and often are) arbitrary, but it is part of the nature of play that you stick to them.

intrinsic motivation

You play because it is play!

Play is closely linked with imagination: a child plays at being a doctor, and of course adults play act as well. This ability to imagine things that are realistic but not real is important not just for literature and art, but also in order to imagine a person on Mars, or a new interface idea. In Chapter 5, we noted the importance of imagination in design. This is not just because of the traditional relationship between imagination and creativity, but also because imagination allows us to harness some of the unconscious knowledge we have including the working of the physical world and social relationships. Imagination is crucial in order to plan ahead, to consider alternatives and to see things form other perspectives.

Play, especially imaginative play, is often considered to be the preserve of childhood and may even be seen as childish in an adult. However, many creativity techniques encourage you to look for 'the child within' to harness the imaginative resources we may have put on hold since childhood – to be childlike is not to be childish. In contrast, computer interfaces often seem totally focused on the rational logical side of our being, and rarely harness our more imaginative and playful side.

Whilst many people despair at our lack of creativity and childlikeness as adults, still it can be argued that the playfulness we have is precisely one of the things that differentiates our human thinking from that of other animals. The pinnacles of human civilisation: science, art and the World Cup are all outpourings of human playfulness. Indeed it is even suggested that we are no so much 'naked apes', but in fact just massively overgrown babies [Br03]

S1.2.5 Left-brain right-brain

Closely related to the issues of rational vs. playful thinking are right brain / left brain discussions. Most human brains have quite specialised areas that are different on the left and right side. In particular, the left side (for right handed people) includes key aspects of language, more logical thinking whereas the right has areas for spatial cognition and more intuitive thinking. The two side of our brain work together, but it is often argued that our adultish society – and our computer interfaces – neglect the power of the right side.

Techniques to help recruit our right-hand brain, often focus on 'confusing' in some way our more rational thinking [Ed99]. For example, if you are trying to sketch a face from a photo. If you turn the photo upside down you will probably make a more accurate sketch. Normally your left-brain says 'face' and the archetypal 'face' gets in the way of drawing the particular face in front of you. Inverting the photo 'turns off' the 'it's a face' reaction and allows you to draw what you actually see. Of course this is draftsmanship not art. Similar techniques include looking upside down through your legs to find something you have lost in a room (yes it really works), or scanning up rather than down list of names in a telephone directory to 'turn off' reading and so ease simple name matching.

Your right visual field goes to the left of your brain and the right visual field goes to the right. So there is some evidence that, for right-handed people, it is better to have logical and linguistic and central focus things on the right had side and more contextual things on the left. Note that the right hand visual field of both eye goes to the left-brain and vice versa; this is often wrongly stated as if the left eye were connected to the right brain. The reason for this slightly odd crossover is that the lens of each eye inverts the image it sees, so in fact the right hand visual field is on the left hand side of the retina – and for that matter the top of your visual field is on the bottom of the retina.

Similarly, some studies of two-handed input devices suggest using the right hand for detailed selection etc., whilst using the left hand for setting context. For example scrolling through a document with a left hand track-pad and then selecting individual words with a right-hand mouse.

S1.2.6 Experience and enchantment

Think of a moment that was emotionally intense, perhaps a special birthday party, the birth of your child, being caught outside in terrifying but awe-inspiring thunderstorm, even that last visit to the dentist. As well as having specific emotions attached to them, they are something more – not just wonderful, frightening, joyful, but an experience: sensual, physical, social, emotional – all working together.

One of the most widely cited concepts on this in HCI is Csikszentmihalyi's flow [Cs90]. This refers to the sense of total engagement and focus reported in many areas including physical sports, intellectual pursuits, playing music and professional activity. It is not the common state of the activity, but refers to peak moments of 'optimal' experience. The crucial elements for flow are 'edginess', operating just at the limits of ones ability, and also attention, focus on a single goal'. However these are part of a longer list of eight characteristics:

1. Clear goal – you know and understand whet you are doing
2. Attention – strong focus on the goal and the activity
3. Loss of self-consciousness – the activity and becomes part of yourself
4. Sense of time – time stands still
5. Feedback – immediate sense of the effects of actions
6. Balance – operating at the edge: challenge and ability balanced
7. Control – a sense of being in personal control
8. Intrinsic reward – the activity has value in itself

Notice how several of these reflect earlier points in this chapter such as intrinsic vs. extrinsic motivation and also design principles we have encountered previously such as the importance of feedback. So it is possible to look at some of these and ask "how can I encourage this in my interface".

However, turning the concept into real design is not straightforward. Partly this is because the popularity of flow as a concept has led to a lot of hype and it is hard to find solid results amongst the plethora of vague material in the academic literature and the web. Also the concept refers to points of occasional peaks of engagement; it is not clear that we want every interaction with a web page to be a 'flow' experience. Finally, flow is not necessarily good in itself. Gamblers certainly experience a sense of flow – they are not in control, but feel in control. If our aim is to separate foolish gamblers from their money then encouraging this false sense of control may be 'good' business, but is it an appropriate aim?

McCarthy and Wright present a different analysis of experience building on works of philosophers and media theorists [MW04]. While flow is more about the momentary experience of the musician caught up in their playing, McCarthy and Wright are more interested in a slightly broader experience such as an evening out at a concert. They identify four main 'threads' of experience:

Sensual thread

Our direct visual, aural, tactile and other sensations are our direct connection to the world and are an essential element of experience. A mechanic may hear the slight vibration that indicates a maladjustment or even impending failure in an engine, and you took the car the garage because it feels odd under your hands. Of course computer processing and telecommunications often break the physical connections that make this possible.

Emotional thread

Even everyday activities are invested with emotion, whether the momentary frustration of a stuck key or the general sense of having a good day. McCarthy and Wright cite a hospital study where nurses, doctors and managers viewed patients with very different values and emotions: people for personal relationships, subjects for professional action, or numbers for efficient allocation.

Compositional thread

Experiences are powerful when they have integrity, when part and whole work together, tell the same story. If you see a cheap banner add on a professional web site the disparity causes tension. As meaning seeking creatures we try to construct sensible and coherent meanings to our experiences; if this may be frustrated or enhanced by appropriate design. As well as the need for coherence of style and message epitomised by the cheap web banner, we can also look at the narrative structure, the way in which each interactive action flows more or less naturally from the last.

Spatio-temporal thread

As noted in the flow characteristics, moments of intense emotion can seem to last for ever, or be over too quickly. Our sense of time is influenced by experiences, but likewise the positioning of an interaction in a particular place at a particular time can influence the sense of experience. This is most obvious in tangible or ubiquitous computing, but even in a plain GUI or web interface when you request or provide information does it 'make sense' for the user at that moment in the interaction?

As well as these threads, which run through an experience, McCarthy and Wright also propose six processes of sense making that can be used as tool to analyse experiences.

Anticipating

We come to any interaction with expectations and these develop as we interact. Note that it is precisely the disappointment of these expectations, which can be a problem if the compositional thread is not well designed. Equally, of there is no sense of novelty or unfolding the interaction is likely to be boring.

Connecting

This is the immediate pre-cognitive sensation – the sense of foreboding entering a darkened room, the pleasure of the touch of fur. This connects very directly to the sensual and emotional threads.

Interpreting

You need to make sense of the experience, allowing you to know what you can do, how things may unfold (linking back to anticipation). More complex emotion such as regret, embarrassment, arise at this level and furthermore problems in being able to interpret the experience may lead to a sense of puzzlement and frustration or possibly wonder and mystery.

Reflecting

Even as we participate and move forward in an experience, we often also look book: was it as we expected, better worse. Perhaps we were frightened but it was still fun. Why did I enjoy that experience, was it the thing itself or perhaps more the companions sharing it?

Appropriating

As you reflect the experience starts to get interwoven into the broader fabric of your life: the previous enjoyable experiences you have had with the same friends, the fairground visits as a child. The experience is not just something that happened, but something that happened to you as part of your life story. (Note this is related to, but different from 'appropriation' as discussed in Chapter [[**XX**]])

Recounting

As we tell back the experience to others or even as we remember it and tell it back to ourselves, we re-experience it in a different way, perhaps highlighting things that were almost unnoticed at the time. We do this in the light of previous reflection, but also continue to reflect on the recalled experience. Of course, of the retelling is in a social situation that situation is itself a new experience and itself becomes integrated with the original story, the mundane event may become more value laden after recounting to others.

As is evident from the connections, the order of the above is not simply linear, but there is also a sense of progression between them.

Not surprisingly notions of 'experience' can be elusive and it is hard to turn the descriptive frameworks of experience into more pragmatic principles for design. However, in areas such as film-making, similar more descriptive frameworks have proved valuable not as formulae for making successful films, but as ways to discuss and to bring to the surface issues. There may not be an obvious rule for 'Connecting', but we can ask ourselves what would be someone's immediate feelings when they pick up a product or visit a web site.

S1.3 Using Affect

S1.3.1 Physiological computing

Physiological computing refers to any non-medical use of computers that includes the sensing of human physiological signs such as heart rate. Very often medical equipment such as hear rate monitors or EEG are used for this, but the needs differ, so research in the area often involves a soldering iron as well as willing volunteers.

The use of physiological measurement in evaluation has already been discussed in Chapter 9, so in this section we are more concerned with its use in the modification or control of applications.

There are two main kinds of use:

biofeedback and control – where the main purpose of the application is to let the user (or often patient) know about some, usually hidden or internal process. Sometimes this is for self-awareness or adaptation of the body state, or it may be used as a form of input device, particularly where there is some form of physical disability.

adaptation – where the application alters itself depending on the sensed emotion of the user, but does not necessarily make this adaptation explicit.

We will look at both of these after considering the kinds of physiological signs and how they can be detected.

Kinds of physiological signs

We can divide human physiological signs into three main categories:

outside – things you can easily detect from a distance such as facial expression, direction of eye gaze, pupil dilation, gait

surface – things that you need to touch the skin to feel such as skin temperature, moisture, muscle pressure and tremble

inside – things that are normally hidden inside the person such as heart rate, hormone levels and brain electrical activity

Of course what a human can normally detect is not the same as what we can detect with electronic sensors. For example, people are quite good at recognising facial expressions, but this requires sophisticated computer algorithms. On the other hand an EEG measures what is going on inside you head through electrical contract with your skin.

Our bodily signs may also not even be normally available to ourselves. For example, we may have to concentrate very hard to feel our own heart rate except when out of breath and have no idea at all of our brain's electrical activity. However other things, like when you have a stomach ache, are very obviously available to you, but only noticeable by others if you look ill in your face.

Also some signs may be more obvious to others than to yourself, for example uncoordinated movements or blinking eyes due to tiredness.

What can be detected

The range of sensors are constantly developing, although more in terms of accuracy, expense and reducing size rather than fundamental capability.

Some of the easiest to measure are connected with general state of body arousal: heart rate, skin conductivity and blood pressure. These can all be measured using standard medical equipment, for example ECG (Electrocardiogram), which measures the electrical activity of the heart. In each case people differ markedly in their baseline measures due to fitness, state of health and mood, so any use of these must involve a period of measurement to establish the baseline, for example, in an adaptive game heart rate could be measured during the introductory sequence (so long as it is not too exciting!). All these measures also tend to be somewhat intrusive, you have sensors strapped or stuck to your body. Recently sensors are developing to measure some of these less intrusively, for example highly sensitive and directional microphones for heart rate or building skin sensors into ordinary buttons. For example, exercise machines in gymnasia often incorporate a heart rate sensor into the hand-grip.

Brain activity can also be detected explicitly either using electrodes attached to the skull of through scanning technology such as MRI (Magnetic Resonance Imaging).  Both are fairly intrusive and only the former is usable in normal situations as scanners tend to be very large and in the case of MRI do not coexist well with computer equipment (or indeed anything metal!). The simplest of these detect basic brainwaves, the levels of electrical activity in different frequency ranges known by Greek letters (see Table [[**BW**]]). (Note, the term 'brainwaves' is discouraged, but is most widely recognised) These frequency bands are associated with different kinds of awareness and attention and whilst with more electrodes can locate activity in different parts of the brain are still quite crude in terns of the information available. Brain scanning gives far richer data of what is happening where, but still too expensive and too intrusive to even be used in evaluation.

	frequency	when observed
alpha	8-12Hz	relaxed concentration, enhanced by meditation
beta	>12Hz	active or even anxious thinking
gamma	26-100Hz	related to problem solving, perception, higher function
delta	<4Hz	in healthy adults: associated with deep sleep
theta	4-8Hz	in adults: drowsiness, light sleep and hypnotic states

Table BW. Frequency ranges for brain activity – 'brainwaves'

While most technologies merely measure arousal, facial expression recognition can be used to detect much more precise emotions. As humans we know that different expressions man different things and this can be detected either by mounting sensors on the face to measure muscle movements, or through computer vision techniques. Of course as humans our ability to distinguish emotions is no perfect and if shown images without context it may be hard to tell the difference, for example, between laughter and anger.

Facial expressions are clearly something we can control and they demonstrate the difference between expressed emotion and felt emotion – we do not always show what we feel inside. However, our faces contain some muscles that are not in our control and whilst it is easy to force a smile with the mouth, it is hard to create an 'eye smile'. Strangely computer detection can sometimes be better than humans at distinguishing false emotions – this is partly because levels of deception are a normal part of human–human relations and it is better for our social relations if we do not know everything that other people are feeling.

Cameras and computer vision can also be used to measure other bodily aspects such as pupil dilation and body gait. The first of these is largely again related to arousal – the reason why candle light is romantic and why Bella Donna eye drops were used by the Egyptians a because they enlarge the pupils and therefore make it appear that you are more attracted to other people ... and there are few things as attractive to another as thinking you are attracted to them! Gait is richer and may reveal tiredness as well as agitation, but of course needs to be disentangled from more purposeful gestures and movement.

Similar to gait is the pressure you exert on a control such as button or joystick. Indeed many games controller have pressure sensitive buttons to allow their use for continuous control and this can be used as an emotion measurement device [SB03]. Of course high pressure may be due to excitement and arousal, or may be due to the need for rapid speed, so some understanding of context is again important for interpretation. The pressure on a controller also includes a muscle tremble and this may be used to detect levels of stress.

Emotion also affects the voice and this could be detected, but of course only in interfaces where voice is normally used, for example, an air traffic controller communicating with pilots. We know of no interface uses of this, but it is certainly used in forms of lie detector.

Biofeedback and brain control

The earliest uses of biofeedback have been in medical of psychological applications where by exposing a normally hidden internal state people find they are able to understand and control it.

Some control may be explicit and medical, for example, a diabetic can monitor insulin levels and use this to decide sugar intake or insulin injections. However, often by simply giving biofeedback it is possible, without knowing quite how, to control a normally autonomous function such as brainwave activity or blood pressure. This can be for therapeutic reasons or to enhance performance, for example, brainwave feedback has been used by musicians prior to performance in order to attain levels of focus and calm [MP08,Ro10].

If an internal physiological signal can be controlled then it becomes a potential input to a computer system and this has been used to provide input capabilities for those who are severely disabled and unable to communicate by other means (see also Chapter 10). However, there is also interest in using this in situations where hands are tied up with other tasks, in particular military applications.

As well as this short-term control, exposing internal signs can be useful for long-term lifestyle monitoring and control. If you have a chronic condition by monitoring it you or your doctor may be able to notice patterns, perhaps discovering a food allergy, or simply knowing which situations exacerbate the condition and thus allow you to selectively choose to avoid them. Long-term monitoring can also detect trends of improvement or decline, for example, if an older person is moving about less than she used to. Traditionally this has been done using diaries and other forms of self reporting, but now it is common practice to use recording apparatus for severe conditions (e.g. small wrist watch device recording heart rhythm). However, ubiquitous sensing technology (see Chapter 20) now makes it possible to record more and more aspects of life: do you type more or less fast than you used to, does you GPS suggest you walk faster or slower.

Affect Adaptions

As well as feeding emotion back to the user, sensed emotion can also be used directly by an application to modify its behaviour.

One application of this is in high stress command and control situations; if a fighter pilot has a very high arousal state, then this might be a bad time to pop up the warning that the undercarriage is due for servicing. Furthermore in such situations automatic systems might take over things the pilot would normally do manually, for example. swapping a fuel feed between wing tanks. This is called dynamic function allocation [HH03] and is a form of task migratability as discussed in Chapter 7. Research in this area has included many forms of sensing including brain activity measured using instrumented helmets.

Similar systems have been proposed for ordinary car drivers, but mostly aimed at detecting sleepiness or lack of attention ... but with a somewhat smaller budget. One option is to use small cameras mounted on the steering wheel or rear-view mirror and to use these to monitor pupil dilation. Another alternative is to use the steering wheel as the patterns of movement start to change as someone begins to become sleepy. Having noted sleepiness the system then needs to somehow alert the driver without startling her. In principle, a steer-by-wire car could temporarily ignore any sudden movements while the driver woke, however solutions which involve taking the driver out of control have huge acceptance problems not to mention legal ones if it ever goes wrong.

Dynamic function allocation has particular interaction issues when control needs to pass back to the user. In the case of dealing with events this is not a problem, the events are simply passed on to the user as normal – from the user's point of view it is just there was a gap with no events occurring; if the user was stressed or had attention focused elsewhere during the period of automated control, then the period without events may not even be noticed. However, of the function that was temporarily automated takes continuous input (that is a status input, see Chapter 17) such as a steering the car, then it is more difficult. Imagine if the steering wheel just did nothing for a few moments, you would probably start to spin the wheel to try and get it to have an effect. If the control was then reverted to you, the car would spin wildly. Instead control has to be passed more slowly and fluidly, perhaps by making the steering wheel increasingly sensitive over a period, or if haptic feedback is available, forcing the steering wheel to turn in the automated direction and gradually reducing the force.

Games have also been an important application area. Sometimes these are 'relax to win games', which are really a form of biofeedback, however there has also been some work on games where the game play is adapted depending on sensed arousal. In an early example, players' heart rate was monitored and the game made more or less stimulating in order to keep arousal at an 'optimum' level [GD05]. The game was a typical shoot 'em up game and the adaptation would make the game more stimulating by increasing the number of enemies whilst making them easier to kill and less dangerous, so the game was more stimulating but not more difficult.

A similar approach was used by researchers at HP's Bristol labs to modify dance music [Cl06]. The system hpDJ used various Bluetooth enabled sensors that measure heart rate, temperature and location to see whether the current music is encouraging a majority of people onto the dance floor or whether they are simply sitting around. Depending on the reactions of the dancers the system dynamically modifies the music even using a genetic algorithm, to generate new mixes.

A different application of the technology is to use physiological responses to work out important moments in the day to enable more effective recording or annotation of 'capture' technology . In a meeting the tone of people's voices might be analysed to determine if they are making an important decision or in the middle of a hot discussion. However,most such applications are in their infancy and there are many problems. One researcher, Cliff Randall from Bristol University, UK, hung a digital camera round his neck and wired it up to a heart monitor. When his heart rate peaked the camera took a photo. The aim was that there would be a series of photographs of the most significant or exciting moments of the day. When he came to look at the captured images he found he had a series of photographs of the tops of flights of stairs.

S1.3.2 Expressing affect and identity

While physiological computing is focused on measuring users' uncontrolled internal responses, other systems instead seek ways to let users explicitly express their emotion.

Explicit expression

Perhaps the most obvious example of this are emoticons in email or instant messaging applications (see also Chapter 19). Note that these represent expressed emotions, not necessarily your felt emotions. You may feel very angry, write something that is more gentle, but still a little sharp and then add a smiley face at the end to soften it further. Emoticons are not just decoration, but for many users especially of IM are an integral part of the experience. Indeed it is not uncommon to see a series of exchanges consisting purely of emoticons.

At a more prosaic level, selecting the colour, size and style of fonts allows the user to express themselves. Even in plain text users will use CAPITALISATION or text smilies to express their emotion to one another ;-). This becomes richer when users are allowed to draw and a combination of writing style and doodles can be very expressive (see Fig. [[**fig**]]).

[[fig - frazzled Alan ]]

Similar techniques to emoticons have been used with human-like avatar faces, so that when the user selects a 'happy' icon, their avatar face smiles by the manipulation of simulated muscle groups [Fa06]. In other systems camera-based analysis looks at particular features of the users face and then uses these to manipulate the corresponding features of a generated face. In some ways these is similar to the physiological computing systems as it involves monitoring of the natural expressions of the face, however, the critical thing here is that the system does not interpret the emotion, but merely passes on aspects of the users expression of it so that the recipient can interpret it.

Expression through action

Ambiguity has been proposed as a powerful design heuristic and one of the key areas is precisely here where the openness of an interface or artefact to be influenced and interpreted allows people to use it in different ways. One suggestion is a door knob that becomes warm when the sound level in the room has been high. This might mean an argument, or simply an excited conversation, the person entering the room cam make their own judgements [GB03].

Somewhere between these very implicit affect cues and the more explicit emoticons is SenToy [HB03]. SenToy is a ragdoll-like creature with blank features and various internal sensors. As you move its arms, shake it, or tip it, the information is fed back into a PC, which can then use it to control a character in a game. However, rather than using the doll to run, jump, kick and punch, instead the doll is used to express emotion. Throwing the doll's arms up expresses surprise, covering the doll's face with its arms denotes fear, shaking it up and down is anger. The expressed emotions are used to determine the emotions of your character (a wizard), which then fights a spell-casting contest with another wizard. Your only control is through the emotions of your wizard.

Like emoticons these are expressed emotions and may be a reflection of your own state, or possibly the emotion you feel would be good for the game play, or somewhere between the two (like our faces in real life). However, the actions you perform on the dolls, although very caricatured, very quickly become 'natural' and after playing for a while it is very hard to put SenToy down!

    

SenToy in use [HB03]

While SenToy is far more complex, it is in some ways is similar to stress relievers, those soft stretch toys you can tie in knots, thump or throw across the room, or the cuddly toys that you stroke to make yourself feel better. The tactile nature of these are very important (remember McCarthy and Wright's sensory thread of experience), however electronic versions of stress relievers have been produced. One, the Splatty, allows you to flick artificial paint at the screen using the mouse [Mi05], just right after a tough telephone call.

Figure SP. The Splatty – venting frustration [Mi05]

Self expression

While all the above mechanisms allow you to express how you feel at a particular moment, others allow you to express who you are. An obvious example of this is the swappable mobile phone shell – do I want to be a plain blue person, a fiery orange or a Snoopy? Personalisation of other features such as ring tones on the phone, or desktop wall paper on a PC sometimes are there to be what you like to gear and see, but also to express to others (and yourself) who you believe yourself to be ... or how you would like other people to perceive you. In terms of Maslow's heirerchy, these are meeting needs of esteem and self-actualisation.

Personal web presence is a major form of this self-expression especially those on sites such as myspace.com. These are places to meet people but also to say who you are, or who you would like to be. Images, colours, layout, music all serve to make each mySpace account different, personal. The strength of this sense of personality despite physical distance was demonstrated by the reaction to the tragic death of Taylor Behl, a 17 year old who was found murdered after being missing for a month. Her many mySpace friends turned her account into a shrine to her memory, but others who simply had heard of the story and visited the site felt they 'knew' her and also posted poignant messages to her.  [Pe06,Wi12]

S1.3.3 Motivation, Persuasion and Pleasure

Motivation

The Splatty described in the last section was designed for workers in call centres. These are hard places to work putting them under continuous stress as operator's performance is constantly measured, and they are also emotionally taxing as most callers are calling because something is wrong. Imagine a call from an irate customer who says their product doesn't work, then after ten minutes, they realise they have not put the battery in. Throughout the call you must be friendly and helpful, yet also try to move the call on as you can see the indicator showing unanswered calls building up. No wonder average time in job is less than 10 months.

For such environments, not only must interfaces work, they must also motivate.

After working for many years designing interfaces for these environments Nicola Millard identified seven C's for motivational user interfaces (MUIs): [Mi05,Mi09]

1. Culture.
2. Content.
3. Control (or choice).
4. Collaboration.
5. Curiosity.
6. Competition.
7. Challenge.

Most of these are self explanatory, but two require special mention. Culture here means the working culture in which the interface is to be placed. Having a slick, attractive, enjoyable interface is no good if the over atmosphere is oppressive, indeed call centres which appear to be identical in all functional respects have markedly different retention rates largely because of the different feelings generated by management. Content also is central, when calls build up you care about getting the job done, the right information needs to where you need it when you need it – motivational user interfaces are not instead of efficiency, on the contrary they demand it.

Interestingly many of the interfaces designed following these principles end up somewhat 'cheesy': bright colours, cartoon characters, bold almost brash graphics. They are the opposite of the greys and blacks of 'designer' interfaces ... but, after a tough call, do you want grey?

While the call centre is particularly challenging, designing for motivation is important in most domains, not least educational software. It is clearly important to engage children so that they continue to use the educational programs (instead of more interesting games), however, motivation is also an important determinant of effective learning. You can force yourself to study, but it is hard to learn if you are not motivated.

Persuasion

In some ways software that encourages learning is also doing something else – persuasion. The area focused on persuasive uses of technology has been named captology by Fogg, who was one of the earliest proponents of the topic [Fo02,Fo03]. This persuasive role is important in many areas including education, health, safety and commerce. Recall that affect can be either a primary or secondary goal of an application. In captology these are termed macrosuasion and microsuasion. Macrosuasion would include a HIV prevention web site that seeks to promote safe lifetstyle and practices. Microsuasion would include a blogging site that encourages you to post by providing feedback when people read your blog.

Designing for persuasion, like designing for any affect is not a science, more an art, with guidelines and frameworks to help structure and inform design. The Functional Triad is a framework developed for captology. In this framework computers take three roles:

Persuasive tools – By making some things easier, they encourage users to do those things. For example, having a heart rate monitor makes it easier to do effective cardio-vascular exercise.

Persuasive media – By enabling simulation they allow people to see the effects of actions or experience things that would otherwise be difficult or dangerous. for example, virtual reality phobia therapy use gradual exposure to simulated situations that are close, but not too close to the real experiences that trigger the phobic reaction.

Persuasive social actors – By acting in the place of a person, they trigger some of the same social responses in users. For example, users may empathise with characters in an interactive story, or more cynically feel obligated to buy when faced with a helpful and attractive virtual sales avatar.

A central part of persuasion is trust. If users or audience suspect that a health campaign is really about drug companies making money they are unlikely to respond. Trust has been an particular problem for eCommerce. We see a shop on the high street, it is there every week, we know people who shop there. An internet shop is different, often we do not know how who to trust or how to know who to trust. This is the power and value of the early internet 'brands' such as Amazon and eBay; they are big enough, well known enough and now old enough to be trusted. Interestingly, these brands are now both effectively 'selling' trust, by selling your product on eBay, or by setting up an Amazon shop your customers feel the security of the security offered by eBay or Amazon. Both of these sites reinforce this by having a (trusted) mechanism for adding customer comments another way of building trust.

Web sites can be made to (appear to) to be more trustworthy by careful design. Signs such as official symbols and logos, privacy policies, terms and conditions all add confidence. Broken scripts, amateur graphics and aggressive commercialism, all diminish it.

In the UK it is interesting to note that the online banks often project images that are less corporate and more friendly than their high street counterparts. This would appear at first to be signals that might look less trustworthy, but in fact they are more in keeping with web style and hence more in keeping with the medium. Furthermore, the on average younger internet-banking customers may have less trust of grey-suited-ness than their elders.

Pleasure

When time and resources are tight then it is clear that the most important thing about a system is functionality; you can achieve what you want to with it even if the interface is bad! Clearly we would also like to be usable, so that things are easy to learn, easy to do, etc. However, once these things are in place the experience of using the system become more important.

Mirroring Maslow's hierarchy of needs, Jordan proposes a hierarchy of consumer needs:

1. Functionality
2. Usabilty
3. Pleasure

He notes that once usability became commonplace it moved from being a 'satisfier' something that makes you happy when you have it, to being a 'dissatisfier', something that makes you unhappy if you do not have it.

Notice an interesting change of words here consumer not user. The word user focuses on the process of use of the product whilst consumer brings to mind individual choice as to whether to use the product at all. In a paid work environment you usually have to use the software provided, whereas a consumer may choose not to purchase or use it at all. Also in the work environment the importance of affect is weak or non-existent, whilst for an individual product the consumer will explicitly value enjoyment and pleasure. (See table TTT)

Table TTT. differences between work and office environment

	work context	consumer context
product choice	corporate	individual
usage choice	must use (but may avoid)	may stop using
explicit importance of affect	low (but shouldn't be!)	high
impact of affect	motivation, efficiency	enjoyment

Of course this doesn't mean that pleasure and other affective features are not important in a work environment (for example, the Splatty in section [**Motivation**]). Indeed usability has long been described in terms of effectiveness, efficiency and satisfaction (e.g. in ISO 9241-11) focused mainly on work environments. However, despite their importance issues such as enjoyment and motivation are often ignored in corporate decisions and are certainly not as central to decision making as they are for consumer products.

Jordan suggests a three step process for designing pleasurable products [Jo02]:

1. understand the consumer
2. know what they want
3. give it to them

The first is most important and like an 'ordinary' usability process (in Chapter 5) involves getting to know who they are, but focusing more strongly ion aspirations, tastes, values rather than intellectual or practical aspects. One of the main tools for recording and communicating this is again the persona. We noted in Chapter 5 that persona was a powerful way to enable the design team to answer questions like "would Betty the warehouse manager understand this?". However, when we want to know what a user (or consumer) would like or enjoy, then it is even more obvious that a rich description is need rather than a plain user profile (age, sex, education, etc.).

S1.3.4 Designing Digital Experience and Fun

McCarthy and Wright's three threads and six processes of sense making (see section [[**30.2.5**]]) were created principally as ways to analyse experience. However, they can also be used as prompts when attempting to create new digital experiences. For example can we build appropriate and suitable expectations (anticipating process), can we help users tell others about the experience (recounting process), can we make a clearer narrative stricture to the interface (compositional thread).

Designing completely novel digital experiences is very challenging, partly because we always bring our personal experience and cultural understanding with us when we come to a new experience, so even if something is novel we will always see it as being 'like' previous things.

Most often the link between past experiences and new products is explicit in the design, e.g. a word processor is 'like' a typewriter, Amazon is 'like' a bookshop. This is clearly useful because it exploits metaphors and knowledge about the functions of the original 'real world' experience. However, the emotional and affective aspects of the original experience will also influence the way people interact and enjoy the new product. This can be positive, if people take forward their enjoyment of browsing in a bookshop to Amazon of the tension of an auction to eBay ... but also can be negative if the real world experience is not good, or if some of the positive emotional expectations are not fulfilled.

Designing digital counterparts to physical experiences is not simple facsimile as many aspects of the physical world cannot be replicated: you cannot experience the feeling of ruffling the pages of a book online. Furthermore simply doing as close a rendition as possible is not sufficient, just as a transliteration of a passage into the words of a different language is not the same as a translation. Indeed, if we make some feature too realistic when others are not, we may risk raising the users expectations and then disappointing them.

However, we can avoid some of these pitfalls by analysing the experiential aspects (feelings and activities) rather than the superficial ones (appearance and form) then using these to guide the design.

Some of the authors were involved in the design of virtual Christmas crackers. These are rather like electronic greetings cards, but are based on crackers. For those who have not come across them, Christmas crackers are small tubes of paper between 8 and 12 inches long (20–30 cm). Inside there are a small toy, a joke or motto and a paper hat. A small strip of card is threaded through, partly coated with gunpowder. When two people at a party pull the cracker, it bursts apart with a small bang from the gunpowder and the contents spill out.

The virtual cracker does not attempt to fully replicate each aspect of the physical characteristics and process of pulling the cracker, but instead seeks to reproduce the experience. To do this the original crackers experience was deconstructed and each aspect of the experience produced in a similar, but sometimes different, way in the new media. Table [[**old-3.1**]] shows the aspects of the experience deconstructed and reconstructed in the virtual cracker.

For example, the cracker contents are hidden inside; no one knows what toy or joke will be inside. Similarly, when you create a virtual cracker you normally cannot see the contents until the recipient has opened it. Even the recipient initially sees a page with just an image of the cracker; it is only after the recipient has clicked on the 'open' icon that the cracker slowly opens and you get to see the joke, web toy and mask.

The mask is also worth looking at. The first potential design was to have a picture of a face with a hat on it – well, it wouldn't rank highly on excitement! The essential feature of the paper hat is that you can dress up. An iconic hat hardly does that. Instead the cracker has a link to a web page with a picture of a mask that you can print, cut out and wear. Even if you don't actually print it out, the fact that you could changes the experience – it is some dressing up you just happen not to have done yet.

A full description of the virtual crackers case study is on the book website at: /e3/casestudy/crackers/

Table 3.1 The crackers experience [101]

	Real cracker	Virtual cracker
Surface elements
Design	Cheap and cheerful	Simple page/graphics
Play	Plastic toy and joke	Web toy and joke
Dressing up	Paper hat	Mask to cut out
Experienced effects
Shared	Offered to another	Sent by email, message
Co-experience	Pulled together	Sender can't see content until opened by recipient
Excitement	Cultural connotations	Recruited expectation
Hiddenness	Contents inside	First page – no contents
Suspense	Pulling cracker	Slow . . . page change
Surprise	Bang (when it works)	WAV file (when it works)

S1.3.5 Aesthetics and Graphic design

When we are designing digital products, whether software or hardware, we are rarely intending to produce works of fine art, however, we often want something that is at least not visually or aurally offensive and hopefully even contributing towards the pleasure of the user.

In larger projects a visual designer may be involved, for example in producing templates for web pages and full designs for main pages, but in small projects the same person may need be coder, information architect and visual designer. Even in larger projects the visual designer may not be available for minor maintenance and updates and so it is easy for an initial design to 'age' or for the content elements of templated interfaces to conflict with the overall design aims. So, while visual design requires its own training and practical experience it is worthwhile all interface designers developing some appreciation for visual design and understanding of how good visual design achieves its purposes.

This is equally true for aural elements, such as the 'crunch' when a file is thrown in the (virtual) wastebasket, and also for the shape and texture of physical products. However, even more than visual design, these are areas where dedicated expertise is usually required, especially for physical design where the costs of mistakes can be very high (e.g. run of 100,000 plastic mouldings). Again in both cases even if you do not create your own sound samples of your own physical designs it is important to understand their aims. For example, if you take a sharp sound intended to grab the user's attention after major faults, and then use the sound after each button press you are likely to have very frustrated (or at least nervous) users.

visual design principles

Much of visual design (and product design) is taught through apprenticeship, and indeed some teachers of graphic design deliberately eschew more analytic or descriptive approaches and focus entirely on exposing students to practice. However, there are some principles of design that are reasonably widely accepted, even if all are subject to exceptions. On the whole rules of design are ways to avoid bad design, but do not create good design, but that is true of most interface design principles and guidelines. Chapter 5 gave some basic guidance on screen design here we will focus on more aesthetic principles.

Several principles focus on the overall look of an image [Ba99,Wa03,Jo00]:

balance

Where is the visual 'weight; of the design? Is it all to one side, or all to the top, or is there balance around the 'centre' of the image. Note that the visual weight is related many things darkness, kind of colour and texture, but the acid test is when you look at the image does it appear to want to fall over. Balance is most easily achieved by symmetry, but asymmetric designs can also be balanced.

dynamism

Too simple balance can appear static and dead. Sometimes this calm effect is what is wanted, but often a sense of dynamism is required with diagonal lines (sloping up to the right in cultures with left-right alphabets) or elements breaking out of frames (see Figure FF below).

harmony

Do the parts seem to fit together with shapes, colours, textures that do not jar. For example, in colour choice you may use complementary colours form opposite sides of the colour wheel, or may choose a single hue, but vary its saturation and brightness.

emphasis

Having some point that stands out, that is the centre of focus. Very often this is more interesting and dynamic of placed off centre with more minor elements balancing it.

unity

Does it seem like one design, or several thrown together? This applies both to individual pages or screens and to an entire system of web site. Text designs often are based on a grid, a basic layout of columns and areas that is common over a whole book or magazine. This is aided by having a consistent visual language, for example, icons with a similar style. If you are designing the screen elements that go with a physical artefact (e.g. mobile phone), then does the visual design of the screens match those of the device?

variety

Too much harmony and unity is boring! Introducing variety can help emphasise the similarities. For example, you might have a similar layout throughout a web site, but use different colours for different pages, especially if you match the tone of the colours (same saturation and lightness in HSL model [[**cross-ref-?chap2?**]]).

Note how the principles complement (or contradict) each other – these are guidance not rules!

Also note that for screen design we may need to accommodate resizable screens, and for web design scrolling may mean that a carefully balanced design is only partially visible at any time. In both cases the design of your system will be juxtaposed with other applications and other websites … will your design look good against a lime green desktop?

The dynamics of interaction are part of the aesthetics. Slight fade-ins or animations may make the transitions after user actions more smooth and emphasise the overall unity of the system – if they reinforce the meaning of the action rather than being superfluous. For example, the MacOS interface distorts windows as they are iconised so that they appear to flow into and out of the icon emphasising the relationship.

Getting timing right is very hard, too fast and things appear jerky, too slow and the system feels sluggish. Do not become so fixated on some interaction effect that would work if only people had a faster machine – it will simply look amateur. Instead, consider what interaction effects are possible and see how they can be used to achieve your design objective.

analysing visual experience

In general, you can use the same principles of decomposition and reconstruction described for crackers to take an ideal visual or interaction design and translate it into something that works on the available platform: remember the golden rule of design (see Chapter 5) "understand your materials".

For example, consider a desired layout in figure FFF.(i). The visual designer has produced this for a paper document. However the diagonal line crossing the frame (Figure FFF.(ii)) makes it hard to reproduce on a web page where the screen can resize radically.

Although the surface form of the design is hard to reproduce, we can try to understand what are experiential aspects of the design. In particular, this is an example of dynamism at work. So one can look at ways of achieving the visual effect of the line breaking the frame and the feeling of dynamism this engenders without having the actual line crossing. Figure GG.(i) shows one way of achieving this. Note how your eye 'joins up' the lines either side of the frame making them appear to cross it. Note also how the multiple smaller lines help build the sense of dynamism, whilst making it less critical that the lines precisely line up with one another when the screen resizes.

Figure GG.  gestalt flow (from [Dx03])

Table DR shows how this features in the original and new image relate to one another and in particular how the experiential elements are preserved even though details of the appearance have changed to accommodate a different medium.

Table DR. deconstruction and reconstruction of the image (from [Dx03])

original image  (figure FF.i)	new image (figure GG.i)
surface elements
strong box	strong box
single thick diagonal	several thin diagonals
actual crossing	not present
experienced effects
breaking boundaries	gestalt feeling of boundary crossing
dynamism by crossing	dynamism by gestalt crossing plus multiple lines suggest movement

form follows function

Note that how the design in Figure FF achieves its dynamism in a very Spartan form – it has a corporate feel and might be for a firm of lawyers who want to express that they get things done (dynamism), but do so in a professional, perhaps even sober, fashion. It might just function for a very sophisticated wine bar, but would not be suitable for a wild nightclub. The visual form creates impressions that should match the values of the company or person it represents.

Other aspects of the visual form may reinforce, distract or conflict with the aim of the interface. For example, we might have a very dominant image, which leads the eye away from the part of the screen with the main message on it.

The 19^th Century architect Louis Sullivan who was responsible for many of the early New York skyscrapers, coined a principle to describe the way he designed:

form always follows function

This is now more widely cited as "form follows function" and often attributed to Sullivan's influential student Frank Lloyd Wright. This principle, which we introduced briefly in Chapter 5, tells us to first think about the function/content/meaning we wish to convey and then let that drive the form of the design.

On screen we have many design elements discussed in Chapter 5: colour, font, alignment, spacing as well as images, sometimes shapes, and in physical designs the three dimensional tactile form and texture.  These elements suggest certain structures and groupings, and also suggest certain things are important because they demand visual attention – are the messages told by this visual form the same as those you wish to convey in the content?

As in the image in Figure FF, the visual form also delivers affective messages. Does the overall design convey the values and message embodied in the content? At a more detailed level graphic or textual elements can 'tell a story' that amplifies the message, for example when discussing typewriten text we might choose a courier font and even slightly misspell to capture the difficulty of correction compared to digital media. This latter is an example of what is known as reflective text, where the graphic form does not just amplify the emotional content, but is intimately related to the message itself. This is related to kinetic text, the animated titles you see in films and video. In fact, both static and moving images can embody 'movement' to enhance their impact [Ba00].

In chapter implementation, we discuss the long standing desire to separate presentaton from fucntionality reflected in, inter alia,the Seeheim model for architecture, and CSS for web pages. However, form follows function suggests that there are limits to this, sometomes content and its presentation are deeply interwoven. This is extends even to the choice of medium, for example, one of the authors was invoved in a project to create a mobile app for local historical archives . As well as giving mobile access, the very fact of it being a mobile app, as opposed to the more common paper booklets, presented messages that the historical archives were more youthful, dynamic, and relevant. As Marshal Mcluhan said, "the medium is the message" [Mc64].

S1.4 Application areas

Eliciting appropriate emotional effects is important in all interfaces, but there are some areas where this is particularly central. We have discussed examples of many of these earlier in this chapter.

S1.4.1 Games

Clearly in games emotion is the primary goal of the system – to make you excited, enthralled, exhilarated depending on the genre of the game. All games use techniques to heighten the emotional impact: soundtrack, graphics, preview material.

Small changes can make a big difference. There is an old video game where you fire coloured bubbles at a wall of coloured bubbles and when your bubble hits two or more similar coloured bubbles they all burst. A desktop variant of this called 'Snood' has different faces associated with the different colours – they even have names, the blue one is called Jake. The more personal, animated version is attractive and addictive in a way that the plain coloured bubbles are not.

 

Snood in action

As well as these more traditional techniques we have seen examples in this chapter where emotion detection can be used to enhance gameplay, either through physiological detection [[**xref**]] or through expressing emotion as in the use of SenToy [[**xref**]]. These are still research tools, but with webcams now ubiquitous and more sensitive pressure sensors of gamepads, it is becoming possible to measure aspects of emotional reactions, so it may not be long before these become part of standard games.

Whilst detection of player's emotion is still in the research stage, games that use characters that exhibit character and emotion is already practical. In particular part of the power of Sims is the way that the life stories of its characters develop as opposed to the traditional game play where the aim is to kill the game characters!

S1.4.2 Toys and companions

Closely related to games are toys such as the Tamagotchi or web pets such as NeoPets. These are designed to elicit string emotional binds in the owner (note owner, not player). They need constant care like a real child or pet if they are to thrive and this demanding nature despite relatively little 'functionality' is what makes them so powerful - the owner invests emotional energy in them and so is committed, so much so that a French woman is reported to have killed a cyclist whilst struggling to 'save' her Tamgotchi [Ba98]. One step beyond electrinic pets, in Japan, home of the Tamagotchi, you can even get a virtual girlfreind, who will get upset if you are late for a date [De11]

Virtual girlfriends – Love Plus on the Nintendo DS 

Children (and adults) have always loved cuddly toys, but with increasingly complex electronics these are beginning to be able to react in similar ways to the Tamagotchi. Some are soft and cuddly and others more robot like. Indeed it is amazing how the crude movements of the Sony AIBO, its head on its side, or wagging tail make it seem alive and loveable.

AIBO

In Japan electronic dolls and PARO a robotic baby seal have become companions for older people when family are far away and space is limited for pets [Gu09,Ha09]. They have a simple repertoire of sounds and behaviour, but are treated as if they were alive and taken out when shopping or visiting like you would a baby or pet.

PARO robotic companion seal

S1.4.3 Education

Barney the dinosaur, one of the earliest sophisticated electronic toys, is partly 'just' a toy but was designed to be an educational toy with accompanying TV shows, DVDs and teachers pamphlets. This and other educational toys build on the principle, often used in non-electronic education, that play is the best form of learning.

Younger children want to learn whatever you give them, but for older children (perhaps dulled by years of schooling) learning seems less fun. As we saw in section [[**30.2.3**]] much educational software make use of extrinsic motivation: they have some 'work' learning with occasional games. More effective are those that make use of intrinsic motivation, where the educational aspects of the software are enjoyable for themselves. Many simulation games fall into this category. For example, the United Nations World Food Programme created an educational video game Food Force (now an open soyrce project [FF12] where you have various missions to bring food supplies through to a crisis area; you learn about the difficulties and importance of food aid and world hunger, but this happens as a natural part of the game play.

Food Force

Learners emotional state is important whether or not they are playing a game. Sometimes computer systems can help this simply by not being human. Often in education students constantly feel under pressure to do things right and feel embarrassed or judged when corrected by a teacher. Computer-based self-assessment can be less stressful because it is not a person and hence does not give a feeling of judgement.

Stress is a problem in education, not just because it is unpleasant, but because it prevents learning. The Yerkes-Dodson curve based on early experimental psychology results back in 1908 suggests that we have optimal performance (including learning) at medium levels of arousal – so students need to be motivated and engaged, but not too stressed nor too excited [YD08]. However, whilst often cited the axes are rarely given values, so in practice it is usually not possible to finely tune to achieve optimal experience, but we can know to avoid the extremes.

Figure YY. Yerkes-Dobson curve

S1.4.4 The Arts

The arts are another place where eliciting emotional response is often the primary goal of the work. Music has been at the forefront of technology from archaic strung instruments and early bellows organs to 1970's synthesised music and of course the electric guitar. Increasingly interactive technologies are being used in the visual and performing arts, both in stage productions with sophisticated projection equipment and also in installations. The former often require complex professional control equipment although these build on the lighting or sound engineer's longstanding array of switches and sliders. Often these are now moving to onscreen versions such as MAX/MSP for MIDI control [Cy12].

In more avant-garde productions, in the gallery or even outdoor installations, the audience/viewer can become a participant in the work. This may be explicit, but also sounds and visuals may be triggered by sensors or may be used to link remote places visually or aurally. The use of technology changes many of the linkages between events and cause and effect making things that were previously hidden visible (for example, using physiological sensing) or making hidden connections [DS05].

S1.4.5 The Home

The home can be a place of frenzied activity: cooking a meal for a hungry family, sorting out bills, however it is also a place of relaxation: reading a book, watching the television. Often too we do the 'work' activities in the home for pleasure: the joy of cooking and eating a leisurely meal. the home is already a place of high technology both devices designed for the 'work' of the home such as the cyclonic vacuum cleaner and devices designed for leisure such as the HiFi. Emotional aspects are important both for the latter as a primary goal and for the former as a secondary goal. Indeed the range of colours and styles of kitchen appliances shows they are not just about basic needs of food preparation, but also about self-expression and enjoyment.

Entertainment systems used to be discrete: the television sat in the corner, the HiFi on a shelf, but increasingly they are both becoming more complex in themselves (DVD, cable or satellite box, hard-disk recorder, interactive TV) and linking with one another and the internet. You can purchase special carousels to hold the separate remote controls for each device. Interfaces to manage this complexity need to be functional, but as important not interfere and if possible enhance the entertainment experience. The success of the iPod was precisely because it was beautiful as well as functional.

cozy remote holder by j-me original design

As the vision of ubiquitous computing (see Chapter 20 becomes reality we may see entertainment devices interact with lighting in the room, perhaps even turn on the kettle when it is near the end of a TV program.  Elements of this are already happening, for example Phillips produce a TV that has lighting units surrounding the set which project colour into the room matching those in the current television scene as if the room was drawn into the TV programme [Ch08].

Fig. PLP. Phillips Ambilight

As well as introducing the notion of ubiquitous computing Mark Weiser coined the term calm technology, for systems that do not assail you with unnecessary information and alerts, moving these to the periphery. This has often been associated with the home as a calm place. Whether those with a young and boisterous family would regard the home as 'calm' is questionable, however certainly technology that is calm seems a good idea instead of the singing cereal packet in Minority Report.

Things in the home don't always need to 'do' something. Bill Gaver has championed the notion of ludic design [Ga01], that is playful rather than functional devices. An example of this is the Drift Table [GB04]. This is a small table with a glass porthole in the centre. As you look into the porthole you see an image of the land projected as if you were in a balloon looking down on the countryside. If you put weight on a side of the table the table drifts over the land in that direction. The table drifts very slowly so you cannot navigate quickly to a place to view it quickly – it is not a glorified road atlas. This sort of technology invites exploration and allows the people living with it to create their own interpretations. One person who had it in his flat for some time ended up piling rocks on the table to make it drift in different directions. Note too that the slow movement also make this calm technology.

The Drift Table [GB04]

S1.4.6 Intimacy

It is increasingly common for couples to be far from one another as jobs take people to different cities and different countries. Even couples living together or close to one another may feel the need to feel close. Standard communication technologies can do this and SMS texting has become a vital part of many people's feelings of being in touch. Text messaging is so powerful for various reasons: the very personal nature of the phone emphasises the intimacy of communication, the asynchronous and almost surreptitious ability to send and receive messages means you can say "thinking of you" at any time or place, and even the shorthand in texting has a sense of playfulness.

In addition to standard technologies a number of specially designed devices have been proposed and prototyped for allowing people to feel more intimate ate a distance. One early device consisted of two rollers that could be placed on desktops at remote places, connecting to each other through the computers and the internet. The rollers each had sensors to detect when they were moved and also motors. When one person moves their roller, the other one moves in synchrony. If both people try to roll their rollers at the same time, they can feel the resistance form the other end, virtually touching one another at a distance [BI98]. Another design 'Scent Whispers' consists of two wirelessly connected brooches: a spider shaped-brooch with a sound sensor and a beetle-shaped one with a small scent spray [Ti03]. When the person with the spider brooch speaks gently on it, the beetle at the other end sprays the scent.

(left) inTouch (right) Scent Whispers

S1.5 Summary

[[** TO DO **]]

Exercises

Further reading

Pi00  Rosalind W. Picard (2000). Affective Computing. MIT Press, Cambridge (Mass.), USA. ISBN: 0262661152

Picard and her group at MIT have been amongst the vanguard in addressing emotion within computing. The book covers many of the theoretical strands of emotion research in greater details than we can in a single chapter as well as potential and actual application areas and ethical issues in affective computing. Also a very good read!

Fo03  B. J. Fogg (2003). Motivating, Influencing and Persuading Users. Chapter 17. Handbook of Human Computer Interaction, Julie A. Jacko and Andrew Sears (eds.). Laurence Earlbaum, New Jersey, USA. pp. 358–370

Fo02  Fogg, B.J. Persuasive Technology: Using Computers To Change What We Think and Do. San Francisco: Morgan Kaufman, 2002. ISBN: 1558606432

If you have the Jacko and Sears handbook available, Figg's chapter gives an introduction to captology and persuasive user interface design. See his book for more detail or the Captology Lab website: http://captology.stanford.edu

MW04  John McCarthy and Peter Wright (2004). Technology as Experience. MIT Press, Cambridge, Mass, USA. ISBN 0-262-13447-0

This book contains a deep cross-disciplinary review of work on experience, culminating in the framework for understanding and analysing experience described in section [[**ref**]]. The framework is applied in detailed case studies covering online shopping, a commercial aircraft pilot and ambulance control.

BO03  M. Blythe, K. Overbeeke, A. Monk and P. Wright (eds.) (2003). Funology: From Usability to Enjoyment. Dordrecht, the Netherlands: Kluwer. ISBN: 1-4020-1252-7

This book was the outcome of a series of workshops on 'computers and fun. and includes chapters on topics including theory of fun and experience, design methods for fun and applications in product design and digital media,

Ed99  Betty Edwards (1999). The New Drawing on the Right Side of the Brain. Harper Collins ISBN: 0874774241

Betty Edwards has been particularly influential in popularising the left–hand right hand debate since her first edition of Drawing on the Right Side of the Brain in 1979 when brain research was still just hinting at hemispherical differences. She proposes a number of exercises which are largely about turning off the left-brain tendency to see things as categories and then draw the category "it's a leaf so it must be green". However, arguably the book is more about learning draughtsmanship, drawing what you really see, rather than art.

Mc02  Chris McManus (2002), Right Hand, Left Hand, Phoenix, London, UK. ISBN 0-75381-355-6

This book covers broadly, and with academic depth, the many left–right differences in our bodies and brains. This is the place to go to get a deep understanding of what is known and what is unknown in this area, not just about the brain as it is, but also how organisms develop asymmetrically and about the advantages and disadvantages of left-handedness.

No04  Donald A. Norman (2005). Emotional Design. Basic Books, ISBN: 0465051367

Norman's book discusses the way emotion can be applied in product deisgn. It was influential in bringing the topic to the attention of the usability community.

References

Ba00 Michelle Bacigalupi (2000). Designing movement in interactive multimedia, Interfaces 44, Autumn 2000, pp.12-15 http://www.bcs.org/upload/pdf/interfaces44.pdf

Ba98  Fred Ballard (1998). Tamagotchi revisited: Driver saves virtual pet, kills cyclist. The Risks Digest. ACM Committee on Computers and Public Policy, 19(67), Tuesday 14 April 1998 (item dated "Thu, 09 Apr 1998 09:54:14 -0500") http://catless.ncl.ac.uk/Risks/19.67.html#subj14

Ba99  Marvin Bartel (1999) Some Ideas About Composition and Design Elements, Principles, and Visual Effects. http://www.goshen.edu/art/ed/Compose.htm </p>

BI98  Brave, S., Ishii, H., and Dahley, A. 1998. Tangible interfaces for remote collaboration and communication. In Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work (Seattle, Washington, United States, November 14 - 18, 1998). CSCW '98. ACM, New York, NY, 169-178. DOI= http://doi.acm.org/10.1145/289444.289491

see also: http://tangible.media.mit.edu/project/intouch/

Br03  Clive Bromhall (2003). The Eternal Child: How evolution has made children of us all. Ebury Press. ISBN: 0091894425

Cs90  Mihaly Csikszentmihalyi (1990). Flow: The Psychology of Optimal Experience. New York: Harper and Row. ISBN 0060920432

Ch08 Stuart Cherry (2008). A visionary experience. Philips Research technology magazine, Issue 31, February 2008, pp.4–7. http://www.research.philips.com/password/download/password_31.pdf

Cl06  Cliff, D. (2006) hpDJ: An automated DJ with floorshow feedback, in O'Hara, K. and Brown, B., Eds. Consuming Music Together: Social and Collaborative Aspects of Music Consumption Technologies, pp. 241-264. Kluwer.

Cy12  Cycling 74. What is Max?. http://cycling74.com/whatismax/

Da00  Antonio Damasio (2000). Descartes' Error: emotion, reason and the human brain. Quill imprint, HarperCollins, New York. ISBN 0-38-072647-5

De11  Danielle Demetriou 'She Feels as Real as My Real Girlfriend': Love Plus captures hearts in Japan. The Telegraph, 11 Dec 2011. http://www.telegraph.co.uk/women/sex/8940765/She-Feels-as-Real-as-My-Real-Girlfriend-Love-Plus-captures-hearts-in-Japan.html

Da72 The Expression of the Emotions in Man and Animals. Charles Darwin, 1872, http://www.gutenberg.org/ebooks/1227

Dx03  Alan Dix (2003). Deconstructing Experience - pulling crackers apart. In Funology: From Usability to Enjoyment. M. Blythe, K. Overbeeke, A. Monk and P. Wright (eds.) Dordrecht, the Netherlands: Kluwer. pp. 165-178. http://alandix.com/academic/papers/deconstruct2003/

DS05  A. Dix, J. Sheridan, S. Reeves, S. Benford and C. O'Malley (2005). Formalising Performative Interaction Proceedings of DSVIS'2005. (Newcastle, UK, 13-15 July 2006). Springer, LNCS 3941, pub. 2006. pp 15-25.

Du62  Guillaume Duchenne (1862). Mécanisme de la Physionomie Humaine (The Mechanism of Human Facial Expression) http://books.google.co.uk/books/about/The_Mechanism_of_Human_Facial_Expression.html?id=a9tjQC7xbNMC

EF78  P. Ekman and W. Friesen. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, 1978.

Fa06  Fabri, M. (2006) Emotionally Expressive Avatars for Collaborative Virtual Environments, PhD Thesis, Leeds Metropolitan University, November 2006

FF12 Food Force 2. http://www.foodforce2.com/[/bibitem]

Ga01  Bill Gaver. Designing for Ludic Aspects of Everyday Life. Special Issue on Ambient Intelligence. ERCIM News No.47, October 2001. http://www.ercim.eu/publication/Ercim_News/enw47/gaver.html

GB03  Gaver, W. W., Beaver, J., and Benford, S. 2003. Ambiguity as a resource for design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA, April 05 - 10, 2003). CHI '03. ACM Press, New York, USA. pp. 233-240. DOI: 10.1145/642611.642653

GB04  William W. Gaver, John Bowers, Andrew Boucher, Hans Gellerson, Sarah Pennington, Albrecht Schmidt, Anthony Steed, Nicholas Villars, and Brendan Walker. 2004. The drift table: designing for ludic engagement. In CHI '04 extended abstracts on Human factors in computing systems(CHI EA '04). ACM, New York, NY, USA, 885-900. DOI=10.1145/985921.985947

GD05  K. Gilleade, A. Dix and J. Allanson (2005). Affective Videogames and Modes of Affective Gaming: Assist Me, Challenge Me, Emote Me Proceedings of DIGRA'2005. http://www.digra.org/dl/display_html?chid=06278.55257.pdf

Gu09 Erico Guizzo (2009). Paro the Robotic Seal Could Diminish Dementia. IEEE Spectrum, May 2009. http://spectrum.ieee.org/robotics/home-robots/paro-the-robotic-seal-could-diminish-dementia

Ha09  Amy Harmin (2009). A Soft Spot for Circuitry. New York Times, Science, July 4, 2010 http://www.nytimes.com/2010/07/05/science/05robot.html

HH03  Hildebrandt, M., & Harrison, M.D. (2003). Putting time (back) into Dynamic Function Allocation. Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting (pp. 488-492).

HB03  Hššk, K., Bullock, A., Paiva, A., Vala, M., Chaves, R., and Prada, R. 2003. FantasyA and SenToy. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA, April 05 - 10, 2003). CHI '03. ACM Press, New York, NY, 804-805. DOI: 10.1145/765891.766002

Hu44  Huizinga, J., 1944. Homo Ludens: A study of the play-element in culture. Routledge and Kegan Paul.

Jo00  Ann Johnston (2000). The Quilters Book of Design. The Quilt Digest Press, Lincolnwood (Chicago), USA. ISBN-0-8442-2660-2

Jo02  Patrick Jordan (2002). How to make brilliant stuff that people love and make big money out of it. Wiley. ISBN: 0-470-84711-5

Ma43  Maslow, A. H. (1943). A Theory of Human Motivation. Psychological Review, 50, 370-396. (avauable at http://psychclassics.yorku.ca/Maslow/motivation.htm)

Ma70  Maslow, A. H. (1970). Motivation and Personality, 2nd. ed., New York, Harper & Row

MP08  Markovska-Simoska S, Pop-Jordanova N, Georgiev D. (2008). Simultaneous EEG and EMG biofeedback for peak performance in musicians. Prilozi. 2008 Jul;29(1):239-52. PMID: 18709013 http://www.ncbi.nlm.nih.gov/pubmed/18709013

Mc64  Marshall McLuhan (1964). Understanding Media: The Extensions of Man. New York: McGraw-Hill

Mi05  Jane Millard (2005). Designing Motivational User Interfaces: Can A Balance Between Effective And Affective User Interface Design Be Used To Motivate Call Centre Advisors?, PhD Thesis, Computing Dept., Lancaster University, UK.

NB01  Ian Noble and Russell Bestley (2001). Experimental Layout. RotoVision, Hove UK.. p. 107

No96 Will Novosedlik, Dumb, Eye – the international review of graphic design 22, Autumn 1996. Illustrated by Stephen Coates. www.eyemagazine.com

Pe06  Janet Pelasara (2006) Love You More: The Taylor Behl Story. William Morrow. ISBN 0061145955

Ro10  Shari Roan (2010). Biofeedback technique eases musicians' anxiety. Los Angeles Times, Health, June 17, 2010

RB98  Roger C. Parker and Patrick Berry (1998). Looking Good in Print, 4th ed. Coriolis, Scottsdale (Arizona), USA. ISBN 1-56604-856-7

SM88  Strack, F, L L Martin, and S Stepper. "Inhibiting and Facilitating Conditions of the Human Smile: a Nonobtrusive Test of the Facial Feedback Hypothesis." Journal of Personality and Social Psychology 54, no. 5 (May 1988): 768–777.

SB03 Jonathan Sykes and Simon Brown. 2003. Affective gaming: measuring emotion through the gamepad. In CHI '03 extended abstracts on Human factors in computing systems (CHI EA '03). ACM, New York, NY, USA, 732-733. DOI=10.1145/765891.765957

Ti03  J. Tillotson (2003). eScent: Jitterbug! Scent By A Wireless Web. Smart Second Skin. http://www.smartsecondskin.com/main/escent_scentwhisper.htm

Wa03  Suzzane Watzmann (2003). Visual Design Principles for Usable Interfaces. Chapter 13. Handbook of Human Computer Interaction, Julie A. Jacko and Andrew Sears (eds.). Laurence Earlbaum, New Jersey, USA. pp. 263–285

Wi12 Wikipedia (2012) Murder of Taylor Behl. Wikipedia (accessed 10/10/2012)

YD08  Robert M. Yerkes and John D. Dodson (1908). The Relation of Strength of Stimulus to Rapidity of Habit-Formation Journal of Comparative Neurology and Psychology, 18, 459-482 (available at "Classics in the History of Psychology", collated by C. Green. http://psychclassics.yorku.ca/Yerkes/Law/)

Other links

• Virtual Crackers. http://www.vfridge.com/crackers/
• http://en.wikipedia.org/wiki/Maslow's_hierarchy_of_needs
• http://www.xenodochy.org/ex/lists/maslow.html
• http://josephdepalma.typepad.com/blog/files/maslow.gif
• The Architecture of the Solomon R. Guggenheim Museum: Form Follows Function http://www.guggenheim.org/new-york/education/school-educator-programs/teacher-resources/arts-curriculum-online?view=item&catid=730&id=120
• about.com Architecture: Master Architects by Jackie Craven, Louis Henri Sullivan, Master Architect, 1856 - 1924, American, "Form ever follows function". http://architecture.about.com/od/greatarchitects/p/sullivan.htm
• All-Wright Site http://allwrightsite.net/
• Phillips HomeLab (page at Noldus). http://www.noldus.com/default/philips-homelab
• http://en.wikipedia.org/wiki/Buridan%27s_as
• SigGraph 2005, CyberFashion. http://psymbiote.org/cyfash/2005/
• Drift Table (YouTube). http://www.youtube.com/watch?v=uRKOypmDDBM
• Conversations on Presence - David Benyon. http://video.google.com/videoplay?docid=-493540447379987007
• Incendiary reflection - modifying ood by manipulating reflections. http://www.shigeodayo.com/incendiary_reflection.html