exercises

20. ubiquitous computing and augmented realities

EXERCISE 20.1

Many researchers are beginning to explore the potential of ubiquitous computing technologies and applications in the home environment. Discuss how the application themes of context-awareness, automated capture and the continuous interaction of everyday computing is relevant to domestic life. Focus your answer on the challenges of family life or life for an aging population.

answer available for tutors only

The home is a very rich domain for exploring ubicomp. Let's look at one example need in a typical home, finding lost objects. We all have occasions in which we cannot locate some important item in the home. How might ubicomp support this problem? As an automated capture problem, you could imagine being able to retraces one's steps in a home back to a time when you last remember having the lost objects. Accessing a recording of your activities at that time might lead you to remember where you placed the item. Of course, nobody will want to retrace their steps in real time, so it becomes a challenge to think how you might efficiently replay the past. Contextual cues are also very important in trying to track down an item. You may recall when and where you last saw an item, and those contextual cues can be used to index into a captured recording of activity. Contextual cues can also help you to flip through recorded memories, so you might be able to enjoy the last time your parents visited your home, or a precious memory the evening before when a young child performed some skill for the first time. From the perspective of continuous interaction, support for interruption seems very important in a household with lots of children and many activities going on at once. How might you provide an aid to remember where a husband and wife left off in a conversation before being interrupted by children?

 

EXERCISE 20.2

Virtual reality has found a number of applications in the games market. Is this a suitable use of such technology? Discuss the possible benefits and disadvantages of exploiting leading-edge technology in a leisure market.

answer

The leisure market is important to the computer industry and, being commercially lucrative, can provide an impetus for development which can then be exploited in other spheres.

If virtual reality takes off in games it will result in the development of cheaper equipment and more robust techniques, since the games customer tends to be young (and not overly rich) but with sophisticated expectations. Indeed, many research projects in universities are currently using equipment developed specifically for games, since the more powerful, general purpose equipment is prohibitively expensive.

Another possible advantage is that it makes the technology familiar and therefore accessible and acceptable.

A possible disadvantage is that the use of this technology in games may trivialize it so that it is not considered as a serious solution to other more weighty problems. However, this seems less likely than the advantages outlined above.

 

EXERCISE 20.3

Data visualization techniques have often increased our comprehension of phenomena: consider the effect that 3D graphics has had on looking at complex models such as those of the atmosphere or the ocean, or in understanding the structure of molecules. What do you consider to be the areas that may benefit most from virtual reality visualization techniques?

answer

This exercise could be expanded to allow students to carry out a brief literature review on the applications of virtual reality techniques. In general they are most promising in areas that cannot be explored in reality, either because it is physically impossible (such as in the manipulation of molecules discussed in Section 20.3.2) or physically dangerous, as in space. Models can be physically manipulated and interacted with using virtual reality methods rather than simply observed. Chemical and medical research, meteorology, oceanography, seismology are all areas that could benefit from such techniques.

 

EXERCISE 20.4 [extra - not in book]

(Cross-refer to Chapter 2) Flight simulators and driving simulators are common in games consoles and amusement arcades.

(i) Home video games typically only have a joystick, mouse or keyboard for input and a normal screen for output. Describe techniques that are used to achieve a sense of immersion in flight and driving simulation programs.

(ii) Compare this with the hardware/software available in arcade simulators.

(iii) If a VR headset were available, what additional cues would be available to enhance the sense of reality.

(iv) There are 'serious' applications of similar VR techniques such as commercial flight simulators or the simulation of the bridge of a large ship using back-projected video. How does this differ from the above?

(v) Describe other serious applications of VR that utilise techniques or equipment beyond those mentioned here.

answer available for tutors only

(i) 3D effects: shadows, occlusion, perspective, etc. Some users will have lifelike driving wheels or joysticks, some with force feedback. The importance of interaction including virtual movement and control of objects.

(ii)The larger screen increases a user's field of view. There may be faster processing to improve interactive feedback. The biggest effect is from better input devices, especially moving motorcycle seats for 'whole body' tracking. Experienced users also note sound effects including under-seat 'woofers' as being of crucial importance.

(iii)Main effects: stereo vision to improve 3D effects. Detection of head movement to give views of different parts of the scene; in some systems, to determine the direction of movement. May also note the importance of fast enough feedback to avoid nausea.

(iv) User may be inside a real physical envirnoment with real controls (e.g. using hydraulic rams to move the cockpit), which make the simulated environment very like reality. Large scale command and control environments, such as the bridge of a ship or a nuclear control room, may include slower, but more complex, multi-user interaction.

(v) Examples include: a VR system at the molecular level, using a 3D input device, the dataglove, to explore complex proteins; virtual surgery, using a computer model of an actual patient, and force feedback, to allow perfection of surgical techniques.

 

EXERCISE 20.5 [extra - not in book]

(Cross-refer to Chapter 19) You have been commissioned by a pharmaceuticals company to advise on the use of advanced visualisation techniques in their company. Suggest possible techniques available for the following:

(i) Visualising their organization chart, which consists of divisions in several countries across the world, with departments, sections and workgroups in each.

(ii) Visualising fluid flows within chemical mixing vats.

(iii) A large database of chemicals with properties.
They may also want to work collaboratively with remote colleagues.

(iv) Describe how multi-user VR could be used with the above and any problems that might arise.

answer available for tutors only

(i) Description of cone trees or other three-dimensional method for hierarchy visualization

(ii) Could talk about ways of using VR techniques such as virtual bubble streams similar to the virtual windtunnel. Problems of visualising the inside of solid 3D objects may be considered and the use of time and interaction to view these in a similar fashion to the Visible Human Body. May also discuss the use of visualisations of the mapping between input conditions and outputs using sliders and coloured histograms as in Influence Explorer.

(iii) Various techniques may be suggested here including traditional charts and histograms (students may like to see 'dancing histograms' linked from chapter 20 on the textbook's web site www.hcibook.com/e3/chaps/ch20/links/). Some students may have seen the 'slice and dice' interfaces used for data-mining. They can also consider various ways of mapping multi-dimensional data into 2 or 3 dimensions (neural networks, simulated gravity, multi-dimensional scaling) and may refer to the techniques used in the web-page visualisations based on 'similarity' (section 19.3.5).

(iv) Expecting mention of avatars, need for 'pointing', problems of different perspectives including visualisation of text labels; would expect some discussion of mutual awareness and perhaps benefits of interaction with shared objects.

 

EXERCISE 20.6 [extra - not in book]

What is the difference between an ambient display and a peripheral display? Suggest some designs for ambient displays for use in a university environment.

answer available for tutors only

Ambience in this sense is a relationship between the display and its nearby environment. Peripheral is a relationship between a display and a user who might view the display. To be peripheral is to be outside the main focus of attention (in the background versus in the foreground). Effective design of an ambient display would allow for it to be in the periphery except when it reflects some important change of information it is reporting.

The design portion of this question is fairly open-ended and could be addressed as a group project. The HCI literature from the late 1990's has had many suggestions for ambient or peripheral displays, and even some work on how to evaluate their effectiveness.

 

EXERCISE 20.7 [extra - not in book]

'Ubiquitous computing suggests that the interface itself can take on the responsibility of locating and serving the user'. In the past, adaptive user interface systems have struggled to get enough information about the user.

Will ubiquitous computing mean that users have to spend all their time telling the computer what they're doing? Consider various current commercial and research ubiquitous computing devices. Find out

(i) to what extent they embody knowledge about the user's location, tasks, etc.

(ii) how they discover this contextual information

(iii) whether you think they do this in a way that is is sufficiently unobtrusive for the user.

(The eClass project in the Future Computing Environments group at Georgia Tech, http://www.cc.gatech.edu/fce/eclass/, studies the impact of ubiquitous computing on education.)

answer available for tutors only

open-ended investigation

 

EXERCISE 20.8 [extra - not in book]

Dean Whitney theme parks are planning a new family exhibit "Lost in Space". Using virtual reality techniques, family groups will be able to fly in a spacecraft and then land on a simulated planet.

(a) Suggest suitable VR technology to give the family a shared experience.
(b) Describe how these can be used to foster a sense of immersion in the virtual environment.

answer available for tutors only

(a)
Room as simulated cockpit, family can sit on real chairs etc., with realistic instruments and seeing each other. Views of outer space projected onto 'windows' - possibly use portholes, smaller cheaper displays! For landing on surface, could dress each member of family up in full body VR suit (or just head goggles with data gloves) and allow them to move and interact in 3D simulated environment with each member of family having an avatar.

(b)
Realistic controls. Force feedback from controls (rattle at take off and landing). Sound effects. Large pistons to move cockpit around. Control and feedback - e.g. press thruster controls and spaceship moves. When in body suits: air blows in windy areas, ability to pick up objects and move around in the virtual space, possibly even alien smells (all bad!).

 

EXERCISE 20.9 [extra - not in book]

A computer games manufacturer is producing a new game called Gulliver.  In this game the players travel through a virtual landscape in which are found various villages.  Some of these villages are occupied by Lilliputians, who are only 6 inches tall; some by ordinary people; and some by Brobdingnagians, who are giants.  The game uses a fully immersive VR headset.  You have been asked to advise the game makers.

  1. If the user ‘stands still’ in the virtual environment, a Lilliputian village that is very close, a normal village some way off and a Brobdingnagian village in the far distance will all look the same apparent size.  What visual cues can the designers use to enable a user to distinguish them?  (Say in your answer if any cues are better at distinguishing the miniature village from the normal one, or the normal one from the giant one.)
  2. How does this change when the user is allowed to move in the environment?
  3. The same manufacturer also produces 3D visualisation software for shopping market analysis.  A particular product lays out spheres in a 3D area where each sphere represents a shopper, the x,y,z coordinates represent age, income and length of stay in the shop and the size of the sphere represents the amount spent.  The resulting pictures are printed in colour, but users report difficulty in seeing the 3D nature of the diagram

    Briefly explain (referring to answers of parts (i) and (ii) where appropriate) why viewing the spheres in 3D is difficult and why an interactive 3D display could help.

answer available for tutors only

  1. N.B. There shuld be a careful distinction to be made between these answers and those of part (ii)
    • NO semantic cues
    • Stereo vision – better for miniature vs. bigger distinctions, but not so good for distinguishing normal from giant because of reduced angular distinction with distance.
    • Blue hazing with distance (especially good for normal vs. giant)
    • Occlusion effects – e.g. large person behind village would mean either person is giant and village is normal or person is normal and village is giant
    • Some possibility of perspective effects if the landscape includes strong linear features
  2. (ii)   Good students will notice that assumptions have to be made about the users’ size and rate of movement in the virtual world.
    • Various parallax effects due to sideways movement to give comparative distance between different villages
    • Rate of approach as one moves towards villages
    • Height of head – looking down on miniature village
    • Comparative size of user’s own virtual hand and body (where present) as the villages are approached
  3. The critical issues will have already been discussed above – this is reapplying them in a different context.

    Difficult because shapes are abstract and different sized spheres, mean size cannot be used as an indication of distance.

    Interactive 3D display allows effects like movement parallax etc., as in part (ii)

 

EXERCISE 20.10 [extra - not in book]

How can depth be represented in a graphical user interface?

answer available for tutors only

In GUI the depth cues often indicate importance/activity level. Such cues are:

  • size:  larger objects appear closer.  
  • interposition:  overlapping objects appear closer.
  • contrast, clarity, brightness, texture:  increasing any of these makes an object appear closer.
  • shadow:  generating a shadow of a foreground object and casting it on the background objects.

Individual exercises

ex.20.1 (tut), ex.20.2 (ans), ex.20.3 (ans), ex.20.4 (tut), ex.20.5 (tut), ex.20.6 (tut), ex.20.7 (open), ex.20.8 (tut), ex.20.9 (tut), ex.20.10 (tut)