ACM Computing Surveys 31(4), December 1999, http://www.acm.org/surveys/Formatting.html. Copyright © 1999 by the Association for Computing Machinery, Inc. See the permissions statement below.

Education: the Unfinished Revolution

Andries van Dam
Brown University     Web: http://www.cs.brown.edu/

Web: http://www.cs.brown.edu/~avd/
Email: avd@cs.brown.edu

presented at

Engelbart's Unfinished Revolution
A Symposium at Stanford University

December 9, 1998

Paul Saffo's Introduction of Andy

I was talking with a gentleman physicist named Edward Teller and he told me that there was something about Hungary. Turned out that there was one high school that a disproportionately large number of scientists who were on the Manhattan project came from. One wonders what was going on in Hungary at the time.

Well, one wonders what was going on at Swarthmore College in the early 1960s because both Ted Nelson and our next presenter, Andy van Dam, both were at Swarthmore at the same time and both have had fascinating collaborations over the years.

Andy is currently, and has been for some time, the T. J. Watson Professor of Technology and Education and a professor of Computer Science at Brown University, a department that he helped to establish.

Like Ted, he is fascinatingly visionary, he has pushed farther on the edges of hypermedia and education than anyone else. He has one disqualification, however, for being a true visionary - he can actually complete projects. But I think when you hear Andy talk you will not hold that small defect against him.

Andy's Presentation

I want to thank you Paul {Paul Saffo, organizer of the Engelbart Symposium} for letting me take part in this wonderful celebration of Doug and of his fabulous team and of the mother of all demos. I don't think we'll see anything like that for decades, probably. But I've got to ask, what have I ever done to you to have to go on after Ted Nelson. You just saw a new medium, folks - standup techno-comedy. {{laughter and clapping}}

I first want to start off by thinking my co-pilot and navigator here, Rosemary Simpson. She's going to be trying to put the pieces of this show together in an article much as she did for the Bush Symposium that we ran at MIT three years ago. And also a host of other folks at Brown who helped me put this pitch together.

Introduction

Well, let me take a look very briefly at a 50,000 foot level at Doug's revolution and ask where has it succeeded.

I'll say that to a first approximation it has succeeded for knowledge workers, from schools on up to industry and to government. We all use word processing - never mind the limitations, we all have some understanding of what a bootstrap community is, we have shared whiteboard software, and we've begun to learn about augmenting human intellect.

But, where the revolution has not at all succeeded as far as I'm concerned, not even in the tiniest bit, is in education. I was at a meeting last week where Rita Colwell, the director of National Science Foundation, spoke and she said that she had only one message: 'its K-12 stupid'. That was a very interesting thing to hear the director of the NSF, a research foundation, say.

Well, I'm here to talk a little bit about why I think its finally time for this revolution to get underway in this particular area.

I've believed that education is the killer app since 1962, when I taught my first computer course to high school students and high school teachers. And the thing that really made me believe that computers could do it was, of course, Sutherland's landmark film on Sketchpad, which got me to go into computer graphics. And I have to say - Ivan is the other father of our revolution.

{{clapping}}

Yes, indeed. Then I got converted to the notion of hypertext by Ted Nelson, whom I met by chance at a technical conference,

and I saw instantly that a combination of these ingredients was the right way to go for multi-modal learning. I contrast multi-modal to uni-modal, which is what I 'enjoyed' in college. For example, I studied fluids and I had a big textbook filled with text and equations and exactly one diagram - it was on the cover. No geometric intuition at all. Ted, of course, had it right again - "if computers are the wave of the future, displays are the surfboards" {from Dream Machines}, and boy are we surfing on the Web today

Related Work

Let's take a look at what some other visionaries have said about various kinds of gadgets that they invented or used in education.

Thomas Edison thought that the answer was the visual medium and the movie projector that he invented.

It did not replace textbooks.

Benjamin Darrow thought it was going to be radio, the 'textbook of the air', a wonderful phrase.

But again, it didn't happen.

Here's what Ted had to say in a wonderful in an anti-classical CAI - the drill-and-kill, "now Johnny", school of educational software - in which he talked about computers and choice and talked about a magical space, the sort of magical space that Ivan visualized, for example, in his AFIP paper in 1965, about virtual reality.

Let the student control the sequence. Responding resources. That's where its at.

Seymour Pappert, also emphasized computers and choice, starting with Logo and constructivist theory and thinking.

He visualized that there would be a deconstruction of curricula in just the sense that Ted talked about in ³No More Teachersı Dirty Looks², and that hasn't happened, for better or for worse. By the way, Mindstorms Lego - get into it. It's amazing.

Alan Kay. The responding resource, the Dynabook, based on simulation, that's at the core of what we're talking about in education today and fortunately Alan's vision is finally being realized today.

We've done a variety of educational software projects since I started on this in 66-67, not all under my direction, and the last one done by a spinoff company. We tried and we, too, failed in a major way.

Now, I'm being cynical, but I want to tell you that I'm being cynical in the Oscar Wilde sense of being a cynic - which is a frustrated idealist. Like Ted, like Alan, like Doug, I really believe in this stuff and sooner or later it really is going to happen.

Reasons for Failure

So, summarizing, here's a little bit of pseudo-code that you can apply to just about every technology.

Except for the real successes - the only two technologies to universally succeed - books and chalkboards, now whiteboards. That's not quite charitable enough - there are partial successes - the use of filmstrip and movies and television, the Apple II, Logo - those all did have an impact. But still, in 1983 in "A Nation at Risk" the authors wrote about "unilateral educational disarmament". In 1997 the Shaw report, which was done by the President's Committee of Advisors on Science and Technology, says that "technology in a period has fundamentally transformed America's offices and factories and retail establishments, but its impact within our nation's classroom has generally been quite modest".

Why? The usual suspects. I won't have time to go through this in detail.

Social problems like disfunctional and non-existing families, poverty, crime, and so on. They dominate. So when we try to think about educational tool fixes we should remember where the hardest problems lie.

Conservatism is a big issue. We do need radical change and conservatism comes very difficult to our society. I love this Machiavelli quote; I've just got to read it to you:

It must be considered that there is nothing more difficult to carry out nor more doubtful of success nor more dangerous to handle than to initiate a new order of things. For the reformer has enemies in all those who profit by the old order and only lukewarm defenders in all those who would profit by the new order, this lukewarmness arising partly from fear of their adversaries, who have the laws in their favor, and partly from the incredulity of mankind who do not truly believe in anything new until they have had the actual experience of it.
Reasons for Hope

Moore's Law, which we've heard a lot about today, makes visual communication possible. It gives us the vividness and immediacy that were predicted for movies and radio, plus - the big one - interaction. Graphing calculators have had impact on the way that Calculus is taught in high schools and in freshman year of college. Dynabooks are finally arriving - not just your laptops but much smaller things designed typically for reading - Rocketbook, Softbook.

Standards - yes, I believe they are a necessary evil - are being gathered based on HTML (Blech), XML - a bit better - so that we can get some interoperability between these electronic books.

And we have the Web and Java. Applets, just-in-time learning, wonderful for lifelong learning, which is what we are all going to be doing, allows us to get away from Ted's tracked curriculum for students and teachers to configure their own courses and units from a set of components that they can find on the Web.

As Howard Rheingold said this morning, it isn't about connecting schools to the Internet, its about teacher education. By the way, have you ever noticed that people say 'teacher training' but 'driver education'... Lets just call it professional development of the teacher corps, and 'its the content, stupid'. We haven't had excitement because we haven't had, to a first approximation, exciting content. That's what we're trying to do a little bit about at Brown as well as other places.

Exploratories

Our focus has been on the use of electronic books and their components.

The electronic book, as far as I'm concerned, should have three pillars:

  • hypertextual information structure

  • knowledge base - the book should 'know' about its content

  • interactive illustrations
It is, of course, an interactive dynamic repository in the way that the pioneers have written about and showed us for years. I won't be talking about the hypermedia aspects today, nor about the knowledge base. I profoundly believe that books should be able to interact with us through intelligent agents, intelligent cognitive tutoring systems, intelligent CAI - still in an early stage but with some notable successes.

I don't work in that area but what I do want to work on is interactive illustrations, hypergrams, dynabook illustrations, whatever you want to call them. If a picture is worth a 1000 words, a moving picture is worth a 1000 static ones, and a truly interactive, user-controlled dynamic picture is worth 1000 ones that you watch passively.

That's what we are working on. We call them exploratories because they combine the best of the exploratorium, where you are free to roam and experiment, with some degree of guidance that you get from a laboratory exercise.

Explorable worlds that have behaviors. Alan Kayıs Vivarium, which, by the way, now has a commercial implementation in NearLife, which is showing at various science museums. Learning by doing, by experimentation, by investigation, by discovery, and of course, embedded in a hypertextual framework.

Team Approach

What we do is work with the Children's Crusade model that I have used since I got to Brown in the mid-60s. Undergraduates are wonderful designers and implementers. We, with more experience, give them some guidance, but we basically let them run. We have lots of constraints on us dealing with the heterogeneity and the fact that largely these are still fairly unformed workers. So, we have to teach these wonderfully gifted amateurs. We have to teach them programming, instructional design pedagogy, user interfaces, graphics, design, and of course the subject matter.

Implementation Strategy

Our implementation strategy is to test on ourselves, to form a Doug Engelbart-style bootstrap community. We use Java Beans and IBM's Visual Age right now as commercially available tools - we no longer have much energy for tool-building.

Goal

The thing I think that makes our approach a little bit different, and I hope unique in fact is that we are really trying to mine our experience and produce a handbook. Somebody this morning, I forget who it was, said "we need to update the community's knowledge in a handbook". Bingo. That's exactly what we're trying to do. Codify.

So far, we've only gotten to the part of dealing with software and process design patterns - notice that I'm using the word 'pattern' in the Alexander sense of pattern, or the Gang-of-Four design patterns book. We're very fond of patterns at Brown and we teach them starting with my freshman Java class because we think they are so important.

We also want to get at some point to learning which kind of pedagogical strategies work and why and incorporate those in patterns.

Demos

Let me tell you a little bit by showing you some of the ones that have been created by our undergraduates this past year. They come typically in clusters and can build on each other in various combinations. They are embedded in hypertext. I demo during the course of my classroom experience and then people experiment afterwards. Lets look at a couple.

The first one is on additive and subtractive color mixing. It could be used just fine with high schoolers or even fifth/sixth graders. Here's some text that describes the concept and here's the applet. {Shows demo}

OK. Let's look next at the signal processing applet. There are about 25 of these. I have the job of taking something that's pretty tough mathematically for sophomores who've not had a course in linear systems, filtering, or anything like that, and try to create what we've been trying to do for a very long time here at Brown - which is to produce some geometric intuition - the Balsa system done by Sedgewick and Brown did algorithms that way. First you get geometric intuition, then you look at mathematics and the code.

{Shows demo}

Design Strategy Handbook

OK. So much for some quick demos. Let me quickly take you through the thing that I think is most important about what we are doing, which is the capturing of experience in the handbook. We have a very preliminary handbook. It has four sections: planning, design, software engineering, and evaluation. We found that when we started instructing our undergraduates, we gave them way too little guidance. Most of them had never designed anything before. So, the templates that we gave them we insufficient. We are now trying to give them a much richer knowledge base on how you do these kinds of applets.

So we'll just show you a very little bit of it. Lets go to design. In there is a storyboard section. This is an example, done by one student for the benefit of her colleagues. Now, lets go to the interface guidelines. An important part of this handbook is "What did we do wrong? How did we evolve?" There's quite a bit of how we do vector mathematics illustrations here. It says 'here's what we tried, and here's what didn't work, and here's what we like now and why - its simpler, easier to understand." {{For an update on work done since this talk was given, see the Exploratories website: http://www.cs.brown.edu/exploratory/}}

Future Prospects

OK. That's what we are doing now. Let's talk a little bit about the future. I think we're tracking a moving target. We're all, to a first approximation, still very much locked into the desktop metaphor and desktop computing. Its brought great benefits, but it also - as was observed earlier - has constrained our thinking. What should we be thinking about even for the near-term future?

Here Thad Starner from MIT's Wearable Computing group is wearing glasses in which there is LCD projection of a perfectly readable VGA resolution screen on his left glass. On the bottom right you see somebody wearing wearable computers. These devices are in research labs transitioning into the real world.

Multi-modal interfaces, where we talk to our computers. Remember Apple's Knowledge Navigator, which I think is still a fabulous vision video. I think that there are lots of things right with it as well as things wrong with it. We'll be able to talk to our computers in limited ways. We are working very heavily at Brown in the combination of gesture - both for two-dimensional and three-dimensional tasks - integrated with speech.

Other people are working on more passive techniques where you are not wearing anything and video is watching you. You know when you are being watched because you go into a specially instrumented room -this is not surveillance.

High bandwidth is coming our way and that will facillitate teleimmersion. Jaron Lanier will, I'm sure, tell you more about that. That's "being there with other people". Collaboration is a theme that you've heard a lot of today.

Here is one vision that I think you ought to be aware of.

It is Henry Fuchs' predominantly, at the University of North Carolina. He's a real pioneer in display and tracking technology and in virtual reality. He and his group are working on office virtual reality, brought to you courtesy of cameras and projectors, all cheap, in the ceiling. The cameras are callibrating every pixel every so many microseconds at every location in the room, constructing a depth map and reflection map of the entire room in realtime, so that as things change in the room that is being captured the maps are updated.. With that kind of information you can project not just on walls, on special devices, but anywhere in the room and what you can do is pipe remote environments to any particular place and create a shared work or play environment. The diagram shows Henry and remote colleagues designing the next generation head-mounted display, which is one of Henry's specialties. Lifelike avatars are being brought to you by this technology. The idea is to make it cheap and not to make it something that is special and lives in research labs only.

So, these are the kinds of environments that we're going to see in the future. Now, I think that that is going to have to force a paradigm shift in the way we think about applications. In architecture you learn that form follows function. I claim that function should follow form, and that when you get these new form factors you really have to think in a new way about what an application is. Applications will not be what they have been today. We need new idioms, metaphors, and techniques for presentation and interaction in these new computing environments. We want to move away from a desktop metaphor that has served us pretty damn well for well over two decades. We're still all collectively living off Doug and company's heritage and its time to get on to other things.

Model interaction in the future, i.e. interaction with and through models and simulation, and Doug's style of collaboration, will be about team-participant interaction, not human-computer interaction. Computers should go away, they should fade into the background. They will do that for exploring, problem solving, social interaction, and various forms of creative expression.

Call to Action

So. last slide. Your mission - certainly mine, but it could be yours should you choose to accept it, is not just to talk about raising the collective IQ of organizations. Lets talk about raising the IQ of kids of all ages and do lifelong learning, starting at a very young and tender ago and going all the way to the very end. Here's what I'd like to do, in addition to everything else that's being done. I'd like there to be some form of national effort to create clip models in all disciplines. A clip model, like clip art, contains everything you need for the task at hand. It needs behavior, it needs simulation capabilities. It's not a silver bullet. There are lots of other ideas - toys that have computers built into them are also terrific learning devicies, but I think that we should be able to get the classic interactive textbooks done for chemistry, physics, mathematics, engineering, biology, social dynamics, archeology, etc., if we have a national effort by which we all engage by submitting to repositories, not through some centrally controlled process.

Postscript

Since this talk was given (December, 1998), I've expanded this vision, made it more concrete, and am working with a small Engelbartian Bootstrap-style team to get the ball rolling. At this point the vision consists of the following:

I think the most important technology-based revolution for the beginning of the new millennium will be in education (finally!). The next-generation learning environment I am envisioning will go considerably beyond today's technologies such as streaming video, canned animation, or the limited information structures afforded by HTML. And, of course, I do not want to be restricted by desktop-based platforms. All of these, while useful, are transitional technologies; I think the future lies in the following kinds of technologies and approaches:

  • Interactive simulations, in the large and in the small, ranging from simple physics and chemistry models to more complex systems of interactive and evolving components. I often call these ³clip models,² by analogy to clip art. What we need for each topic is a family of models at various levels of detail and sophistication to suit different age levels, backgrounds, and needs for information.

  • Customizable cognitive tutors that can work with both individuals and groups.

  • Collaborative and networked experiences and projects, which may be based on massive multi-player game engines. These projects will focus on enabling learners to construct new objects, behaviors, and ecologies. I am thinking, for example, of next-generation MMP SimCity-style environments such as SimBiology, SimChemistry, etc.
I visualize housing a repository of learning objects embodying these approaches in a digital library. They will be delivered on a variety of platforms, including heterogeneous, intercommunicating mixtures of desktop computers, ultraportable wireless devices, smart-room technologies, and immersive VR.

Embedded assessment for a nearly continuous evaluation should be an important component of most new types of learning materials.

Many of these ideas are, of course, elaborations of visions expounded earlier by pioneers such as Alan Kay, Ted Nelson, and Seymour Papert. My hope is that we can not only realize these visions but also extend them to provide a flexible and extensible foundation for lifelong education.

The Microsoft Research TAB and folks in MSR are brainstorming means of establishing a consortium of companies, government agencies, and foundations that could fund a Grand Challenge-style set of interdisciplinary research projects in educational technology. The aim is to explore various methodologies for creating exemplars of next-generation educational materials, tools, and processes.

Intertwined and mutually reinforcing research areas that should be supported include:

  • Next-generation educational content research, as described above. I hope many exemplar experimental courses will be created, starting with science, math, engineering and technology.

  • Learning science research applicable to this type of Web-based educational content.

  • Tools research and development, including authoring environments, delivery mechanisms, and digital library technologies to index and access the new content.
I have been working in the area of educational software since the 1960s and have been continuously disappointed and frustrated by the gap between hype and reality. Among the many reasons for slow progress have been the lack of a global distribution mechanism, (now at least partially solved by the Web), insufficient resources, and too few highly qualified researchers and developers. By finally focusing attention and effort through a Grand Challenge-scope project, we intend to achieve breakthroughs in this area of enormous national significance.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept, ACM Inc., fax +1 (212) 869-0481, or permissions@acm.org.