Ben Shneiderman
Department of Computer Science
University of Maryland
College Park, MD 20742

email: ben@cs.umd.edu (301) 405-2680 phone (301) 405-6707 fax

Advanced Graphic User Interfaces: Elastic and Tightly Coupled Windows

Windows with 40-60 icons or scrolling lists with 20-40 are inadequate in dealing with the complex tasks that users increasingly face. Advancing hardware, software, and networking technology have raised expectations for users of Geographical Information Systems (GIS), 3-D graphics tools, information directories, scientific visualization, medical image databases, desktop publishing, programming environments, network management, video or animation editing, and other domains. These domain experts are motivated users who are attempting more ambitious projects that demand rapid processing and access to large amounts of visual information, but unfortunately the window managers in graphical user interfaces (GUIs) have not kept up with the users needs. Advantages of large screens and fast displays are lost or misused, leading to confusion, poor user performance, frustration, and missed opportunities.

The computer industry appears to have inadvertently created a de facto standard based on 1984 era hardware and window managers (Macintosh, Windows, OS/2, Motif). This lack of innovation has left most users with the tedious job of manipulating one window at a time. Too much window housekeeping distracts them from their professional tasks and restricts what they can accomplish. The current overlapping independent windows paradigm has been shown to have problems, but viable improvements have been slow to emerge. We believe that important new research avenues are open for coordination within complex interfaces, specification methods for dynamic systems, design principles such as tight coupling, and improved visual information presentation.

Existing principles such as direct manipulation have been widely applied in word processors, spreadsheets, drawing tools, and many other environments:

The benefits of direct manipulation are: control/display integration to simplify usage and conserve screen space, and less syntax to reduce error rates, speed learning and increase retention. The use of properly designed visual representations helps to make the operation more comprehensible, predictable, and controllable, thus increasing the users's willingness to take responsibility for their actions. Of course, there are concerns such as: the possible need for increased system resources, some actions may be cumbersome, macro techniques are still weak, history/tracing may be difficult, and visually impaired users have more difficulty.

The current GUIs are still quite primitive and poorly designed to take advantage of the remarkable human visual perceptual system and large, rapid, and high resolution computer displays. It seems increasingly archaic to see only 40-60 icons on the screen, deal with the cluttered desktop of overlapping windows, and waste time with unnecessary window housekeeping, when appealing alternatives are beginning to appear in research prototypes. Our first proposal is for Elastic Windows in which multi-window operations are achieved by issuing operations on a hierarchically organized group of windows in a space-filling tiled layout. We have developed multi-window operations like Hook, Pump, Minimize, Restore, Move and Relocate to allow users to rapidly restructure their work environment. We claim that these multi-window operations and the tiled layout decrease the cognitive load on users. Users found our prototype system to be comprehensible and enjoyable as they playfully explored the way multiple windows are reshaped. Our second proposal is for Tightly Coupled Windows in which relationships between the contents of windows are easily specified and changed. For example, synchronized scrolling would allow a user to specify two or more windows to be scrolled with only a single user action. In hierarchical browsing selection of a chapter title in a table of contents window causes the full text to be scrolled to the chapter in a second window. Some of these benefits can be achieved through proper use of Netscape frames.

Information visualization: Dynamic queries, starfield displays, and LifeLines

The future of user interfaces is in the direction of larger, higher resolution screens, that present perceptually-rich and information-abundant displays. With such designs, the worrisome flood of information can be turned into a productive river of knowledge. Our experience during the past five years has been that visual query formulation and visual display of results can be combined with the successful strategies of direct manipulation. Human perceptual skills are are quite remarkable and largely underutilized in current information and computing systems. Based on this insight, we developed dynamic queries, starfield displays, treemaps, treebrowsers, and a variety of widgets to present, search, browse, filter, and compare rich information spaces.

The dynamic queries are animated user-controlled displays that show information in response to movements of sliders, buttons, maps, or other widgets. For example, in the HomeFinder the users see points of light on a map representing homes for sale. As they shift sliders for the price, number of bedrooms, etc. the points of light come and go within 100 milliseconds, offering a quick understanding of how many and where suitable homes are being sold. Clicking on a point of light produces a full description and, potentially, a picture of the house.

The starfield display was created for the FilmFinder, which provided visual access to a database of films. The films were arranged as color coded rectangles along the x-axis by the production year and along the y-axis by popularity. Recent popular films were in the upper right hand corner. Zoombars (a variant of scroll bars) enabled users to zoom-in in milliseconds on the desired region. When less than 25 films were on the screen, the film titles appeared and when the users clicked on a film's rectangle, a dialog box would appear giving full information and an image from the film. The commercial version of starfield displays will be available late in 1996 from IVEE Development (www.ivee.com).

In our LifeLines prototype, we applied multiple timeline representations to personal histories such as medical records. Horizontal and vertical zooming, focusing, and filtering enabled us to represent complex histories and support exploration by clicking on timelines to get detailed information.

There are many visual alternatives but the basic principle for browsing and searching might be summarized as the Visual Information Seeking Mantra:

Overview first, zoom and filter, then details-on- demand

In several projects I found myself rediscovering this principle and therefore wrote it down and highlighted it as a continuing reminder. If we can design systems with effective visual displays, direct manipulation interfaces, and dynamic queries then users will be able to responsibly and confidently take on even more ambitious tasks.

The computing industry and the research community have the chance to move ahead with a new generation of systems. In addition to our work, research on information visualization is emerging at key sites such as Georgia Tech's Graphics Visualization and Usability Center, Xerox's Palo Alto Research Center, and Lucent Technologies (formerly AT&T Bell Labs) in Napierville, IL.

References to our research

Ahlberg, C., Williamson, C., and Shneiderman, B., Dynamic queries for information exploration: An implementation and evaluation, Proc. ACM CHI'92: Human Factors in Computing Systems, (1992), 619- 626.

Ahlberg, C. and Shneiderman, B., Visual Information Seeking: Tight coupling of dynamic query filters with starfield displays , Proc. of ACM CHI94 Conference, (April 1994), 313-317 + color plates.

Ahlberg, C. and Shneiderman, B., AlphaSlider: A compact and rapid selector, Proc. of ACM CHI94 Conference, (April 1994), 365-371.

Asahi, T., Turo, D., and Shneiderman, B., Using treemaps to visualize the analytic hierarchy process, Information Systems Research 6, 4 (December 1995), 357-375. [Postcript file] [Text only]

Doan, K., Plaisant, C., and Shneiderman, B., Query previews for networked information services, Proc. Advanced Digital Libraries Conference (May 1996). [Postcript file] [Abstract file] [Text only] [HTML file]

Johnson, B. and Shneiderman, B., Tree-maps: A space filling approach to the visualization of hierarchical information structures, Proc. IEEE Visualization '91 (October 1991), 284-291. [Postcript file] [Abstract file]

Kandogan, E. and Shneiderman, B., Elastic windows: Improved spatial layout and rapid multiple window operations, Proc. Advanced Visual Interfaces Conference '96, ACM Press, New York, NY (May 1996). [Postcript file] [Abstract file] [Text only]

North, C., Shneiderman, B., and Plaisant, C., User controlled overviews of an image library: A case study of the Visible Human, Proc. 1st ACM International Conference on Digital Libraries (March 1996), 74-82. [Postcript file] [Abstract file] [Text only] [HTML file]

Plaisant, C., Carr, D., and Shneiderman, B., Image-browser taxonomy and guidelines for designers, IEEE Software 12, 2 (March 1995), 21-32.

Plaisant, C., Rose, A., Milash, B., Widoff, S., and Shneiderman, B., LifeLines: Visualizing personal histories, Proc. of ACM CHI'96 Conference (April 1996), 221-227, 518. [Postcript file] [Abstract file] [Text only] [HTML file]

Shneiderman, B., Designing the User Interface: Strategies for Effective Human-Computer Interaction, Second Edition, Addison- Wesley Publ. Co., Reading, MA (1992).

Shneiderman, B., Beyond intelligent machines: Just Do It!, IEEE Software 10, 1 (January 1993), 100-103. [Postcript file] [Text only]

Shneiderman, B., Dynamic queries for visual information seeking, IEEE Software 11, 6 (1994), 70-77.

Williamson, C., and Shneiderman, B., 1992. The Dynamic HomeFinder: Evaluating dynamic queries in a real-estate information exploration system, Proc. ACM SIGIR'92 Conference, Copenhagen, Denmark, (June 1992), 338-346. Reprinted in Shneiderman, B. (Editor), Sparks of Innovation in Human-Computer Interaction, Ablex Publishers, Norwood, NJ, (1993), 295-307. [Postcript file] [Abstract file] [Text only]