Database Technology At A Cross-Roads

Umeshwar Dayal

Hewlett-Packard Laboratories
1501 Page Mill Road, 1U-4
Palo Alto, CA 94304
dayal@hpl.hp.com

Data management technology is at a crossroads. On the one hand, the database systems business is phenomenally successful worldwide. Database research has made important contributions to this success: the relational model, ad hoc queries, and the transaction abstraction, are all essential ingredients of commercial data processing. It is tempting now to "polish the round ball", to focus on engineering that is best done by better equipped product development teams: better optimization tricks for subsets of SQL92, more parallel join algorithms, active or object extensions to existing products, CORBA/COM harmonization, and the like.

On the other hand, there are many disruptive forces and transformations occurring in the world around us that are forcing us to rethink data management technologies and to apply them in innovative ways to new application areas. The confluence of global interconnectivity, powerful computing, low-cost on-line storage, and interface technologies are creating many potential opportunities.

Information has become pervasive, thanks to the explosive growth of the Internet and its reach out from the enterprise to the telecommuter and mobile professional to the consumer at home. Enterprises want to use the global information infrastructure for finding, disseminating, sharing, and acting on information; for mining the data on their corporate Intranets to gain competitive advantage; for supporting alternative work/life practices for their employees; and for automating their business processes. Individuals want access to information, resources, and community services for work, recreation, or education. There are proposals to put all of the authored works of mankind on the World Wide Web. This represents a mindboggling quantity and variety of information. How are we going to find the information we want, when we want it, and in a form that we can use?

There are many data and information management problems that will be of enormous value if we can solve them. A few examples:

efficiently finding relevant information from a wide variety of sources on the global information infrastructure
managing the acquisition, storage, and mining of earth sciences data (e.g., tracking a hurricane and predicting its path before it reaches land)
approximate queries over multiple multimedia repositories (e.g., find news articles, photographs, and videoclips of recent forest fires)
telepresence (e.g., home health care, remote consultation and diagnostics, and telesurgery)
managing 10,000-node systems to meet quality of service guarantees
automating complex inter-enterprise business processes (e.g., supply-chain management, collaborative engineering in a virtual enterprise).

Each of these requires the solution of many constituent problems:

modelling and storing vast quantities of complex data of rich types;
annotating and cross-indexing information in multiple repositories;
capturing metadata to enable search and retrieval;
managing and mapping ontologies;
agents for personalized search, filtering, and mining;
streaming and rendering data with real-time constraints;
security and privacy issues;
pricing policies and payment protocols;
new transaction models to support complex business processes and modes of interaction; and
guranteeing quality of service attributes (reliability, scalability, manageability, performance).

As database researchers, we have a lot to contribute. However, these problems will be only solved by "out of the box" thinking. The old compartmentalized notions of what a database system is and what constitutes database research have to change quite radically. We have to carefully unbundle the services provided by today's monolithic database management systems, incorporate techniques from other disciplines, and enable the selective rebundling of services to match an application's functional and performance requirements. There is a lot of scope for conceptual and algorithmic work, for building prototypes, and for carrying technology into the real world of products and services. Will we rise to the challenge?

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept, ACM Inc., fax +1 (212) 869-0481, or permissions@acm.org.

Umeshwar Dayal <dayal@hpl.hp.com>