The WIMP human–computer interface may have an uninspiring name, but Windows, Icons, Menus, and Pointing (WIMP) devices have dominated computing for some 15 years

Subject:ManagementPrice: Bought3

Share With

The WIMP human–computer interface may have an uninspiring name, but Windows, Icons, Menus, and Pointing (WIMP) devices have dominated computing for some 15 years. The keyboard, mouse, and display screen have served users extraordinarily well. Now the hegemony of WIMP may be coming to an end, say developers of technologies based on human touch and gesture. For evidence, look no further than Apple’s iPhone. From a human-interface point of view, the combined display and input capabilities of the iPhone’s screen, which can be manipulated by multiple fingers in a variety of intuitive touches and gestures, is nothing short of revolutionary. The iPhone isn’t the only commercial device to take the human–computer interface to a new level. The Microsoft Surface computer puts input and output devices in a large tabletop device that can accommodate touches and gestures and even recognize physical objects laid on it. In addition, the DiamondTouch Table from Mitsubishi is a touch- and gesture-activated display that supports small-group collaboration. It can even tell who is touching it. These devices point the way toward an upcoming era of more natural and intuitive interaction between human and machine. Robert Jacob, a computer science professor at Tufts University, says touch is just one component of a booming field of research on post-WIMP interfaces, a broad coalition of technologies he calls reality-based interaction. Those technologies include virtual reality, contextaware computing, perceptual and affective computing, and tangible interaction, in which physical objects are recognized directly by a computer. “What’s similar about all these interfaces is that they are more like the real world, Jacob says. For example, the iPhone “uses gestures you know how to do right away, such as touching two fingers to an image or application, then pulling them apart to zoom in or pinching them together to zoom out.” These actions have also found their way into the iPod Touch and the track pad of the new MacBook Air. “Just think of the brain cells you don’t have to devote to remembering the syntax of the user interface! You can devote those brain cells to the job you are trying to do.” In particular, he says, the ability of the iPhone to handle multiple touches at once is a huge leap past the single-touch technology that dominates in traditional touch applications such as ATMs. Although they have not gotten much traction in the marketplace yet, advanced touch technologies from IBM may point a way to the future. In its Everywhere Displays Project, IBM mounts projectors in one or more parts of an ordinary room and projects images of touch screens onto ordinary surfaces, such as tables, walls, or the floor. Video cameras capture images of users touching various parts of the surfaces and send that information for interpretation by a computer. The touch screens contain no electronics—indeed, no computer parts at all—so they can be easily moved and reconfigured. A variation on that concept has been deployed by a wine store in Germany, says Claudio Pinhanez at IBM Research. The METRO Future Store in Rheinberg has a kiosk that enables customers to get information about the wines the store stocks. “But the store’s inventory was so vast customers often had trouble finding the particular wine they wanted on the shelf. They often ended up buying a low-margin wine in a nearby bin of sales specials, “ Pinhanez says. Now the kiosk contains a “show me” button that, when pressed, shines a spotlight on the floor in front of the chosen item. IBM is also working on a prototype system for grocery stores that might, for example, illuminate a circle on the floor that asks, Do you want to take the first steps toward more fiber in your diet? If the customer touches “yes” with his foot, the system projects footsteps to the appropriate products, such as high-fiber cereal. “Then you could make the cereal box itself interactive,” says Pinhanez. “You touch it, and the system would project information about that box on a panel above the shelf.” Asked if interactive cereal boxes might be a solution in search of a problem, Pinhanez says, “The point is, with projection and camera technology you can transform any everyday object into a touch screen.” He says alternatives that are often discussed (e.g., a store system that talks to customers through their handheld devices) are hard to implement because of a lack of standards for the technology. Microsoft is working with several commercial partners, including Starwood Hotels & Resorts, which owns the prestigious Sheraton, W, Westin, and Méridien brands, among others, to introduce Surface. It will initially target leisure, entertainment, and retail applications, says Mark Bolger, director of marketing for Surface Computing. For example, he says, one could imagine a hotel guest using a virtual concierge in a Surface computer in the lobby to manipulate maps, photos, restaurant menus, and theater information. Some researchers say that a logical extension of touch technology is gesture recognition, by which a system recognizes hand or finger movements across a screen or close to it without requiring an actual touch. “Our technology is halfway there,” IBM’s Pinhanez says, “because we recognize the gesture of touching rather than the occlusion of a particular area. You can go over buttons without triggering them.” Patrick Baudisch at Microsoft Research says the Microsoft prototypes can already act on finger gestures, with the system recognizing finger motions, as well as positions, and understanding the meaning of different numbers of fingers. For example, the motion of one finger is seen as equivalent to a mouse movement, a finger touch is interpreted as a click, and two fingers touching and moving is seen as a scroll command. Touch technology in its many variations is an idea whose time has come. “It’s been around a long time, but traditionally

in niche markets. The technology was more expensive, and there were ergonomic problems,” he says. “But it’s all kind of coming together right now.” The rise of mobile devices is a big catalyst, because the devices are getting smaller and their screens are getting bigger. When a screen covers the entire device, there is no room for conventional buttons, which makes it necessary to have other types of interaction (e.g., voice). Of course, researchers and inventors have envisioned even larger touch displays, including whole interactive walls. A quick YouTube search for “multitouch wall” shows that a number of these fascinating devices have reached the prototype stage, causing multitudes at technology conferences to be entranced. Experts predict, however, that this is just the beginning. Pradeep Khosla, professor of electrical and computer engineering and robotics at Carnegie Mellon University in Pittsburgh, says touch technology will proliferate, but not by itself. “When we talk face to face, I make eye gestures, face gestures, hand gestures, and somehow you interpret them all to understand what I am saying. I think that’s where we are headed,” he says. “There is room for all these things, and multimodal gestures will be the future.” Bill Buxton, a researcher at Microsoft, also anticipates a fusion of different interaction technologies. “Touch now may be where the mouse was in about 1983,” Buxton says. “People now understand there is something interesting here that’s different. But I don’t think we yet know what that difference could lead to. Until just one or two years ago there was a real separation between input devices and output devices. A display was a display and a mouse was a mouse.” “There’s been this notion that less is more—you try to get less and less stuff to reduce complexity,” he says. “But there’s this other view that more is actually less—more of the right stuff in the right place, and complexity disappears.” In the office of the future, Buxton predicts, desktop computers might be much the same as they are today. “But you can just throw stuff, with the mouse or a gesture, up onto a wall or whiteboard and then work with it with your hands by touch and gesture standing up. Then you’ll just pull things into your mobile and have this surface in your hand. The mobile, the wall, the desktop—they are all suitable for different purposes.” Will that be the end of the WIMP interface? Tufts University’s Jacob advises users not to discard their keyboards and mice anytime soon. “They really are extremely good,” he says. “WIMP almost completely dislodged the commandline interface. The WIMP interface was such a good invention that people just kind of stopped there, but I can’t believe it’s the end of the road forever.” Buxton agrees. “WIMP is the standard interface going back 20-plus years, and all the applications have been built around that,” he says. “The challenge is, without throwing the baby out with the bath, how do we reap the benefits of these new approaches while preserving the best parts of the things that exist?”

CASE STUDY QUESTIONS	REAL WORLD ACTIVITIES
1. What benefits may Starwood Hotels derive from the introduction of touch-screen technology as noted in the case? What possible disruptions may occur as a result. Provide several examples of each.	1. Most of the fame attached to the iPhone has resulted from individual, end-user applications. How could companies use the iPhone as a platform for commercial use? Break into small groups and brainstorm some possible uses of the technology, as well as what benefits organizations can derive from them. Then prepare a presentation to share your ideas with the class.
2. Bill Buxton of Microsoft stated that “[t]ouch now may be where the mouse was in about 1983.” What do you make of his comments, and what do you think it would take for touch technology to displace the WIMP interface? Justify your answer.	2. Information technology advances rapidly, and touch screen is no exception. Go online and search for developments more recent than those mentioned in the case. What new large-scale (i.e., wall-sized) applications could you find? Prepare a report comparing new developments with the examples mentioned here.
3. Is advanced touch-screen technology really a solution in search of a problem? Do you agree with this statement? Why or why not?