Building and Using IOLIB

Christian Convey, cjc@cs.brown.edu, Sept. 3, 2002.

Building IOLIB

I haven't done much with Makefiles before, so my approaches may be pretty unorthodox. If something looks done strangely to you, don't immediately assume I had a good reason for doing it in a non-standard way.

Untar the source code. The top-level directory produced by the tarfile is "sm/". We'll refer to the parent directory of "sm/" as tar-directory.
Edit the file tar-directory/sm/src/makedefs.mk, as follows:

Modify the PROJBASE variable to = tar-directory/sm/src
Modify the TARGETBASE variable to = tar-directory/sm

cd to tar-directory/sm/src
run make

If all goes well, you should see the directory tar-directory/bin/ populated with valid symbolic links to a set of test/benchmark programs.

Tools versions

I've only tried building this in my local environment, which has the following relevant versions:

Intel Celeron A 300MHz processor
Mandrake Linux 8.0, with kernel version 2.4.8-26mdk
g++ 3.1.1

Packages

I did choose to break the code into "packages", to keep the complexity of code from getting overwhelming. The packages are as follows:

common The "common" package has all of the code that I figured wasn't Storage Manager specific. I pulled such code into a separate package mostly to keep myself from being overwhelmed by having too much code to consider at one time.
iolib This has the code for interfacing with the file system. If you look at the code strucutre diagram that I handed out in August 2002, this covers everything from the bottom of the diagram up through the AsynchPageDir class.
sm This is where I figured I'd put the remainder of the Storage Manager's code. However, since I never got to implement it, this package is empty.

Directory Structure

Like I said earlier - I may have done some things oddly out of ignorance. There's nothing sacred about how I approached this.

sm/bin/ This is where the programs exported by the Storage Manager code belong. This includes not only whatever interesting utilities we might dream up, but also unit tests and benchmark programs for the various internal components of hte Storage Manager.
sm/include/ This is where we give the header files for all the code that the Storage Manager shares with the outside world. This includes the helper classes / functions that aren't specific to the Storage Manager, which which I still thought other parts of the Aurora system might like to use. There should probably be one .H file here for each .o file that appears in the directory sm/obj/.
sm/lib/ This is where I intended the library containing the Storage Manager's code to appear. It's currently empty because I never worked out how I wanted to produce the library(ies).
sm/obj/ This directory can probably go away if you ever produce a library containing the Storage Manager's code. This contains all of the .o files from all of the packages in the Storage Manager. If you have the Storage Manager ship as a library, no one would need the .o files that appear in this directory.
sm/src/ The root of the directory tree containing all of the source code and header files for the Storage Manager.
sm/src/common/cpp/ .C / Makefile files for the "common" package.
sm/src/common/include/ Header files for the "common" package that are needed by those trying to use the "common" package's exposed functionality.
sm/src/common/int_include/ Header files for the "common" package that are purely for internal use, and aren't needed by those trying to use the package's exposed functionality.
sm/src/iolib/cpp/ .C / Makefile files for the "iolib" package.
sm/src/iolib/include/ Header files for the "iolib" package that are needed by those trying to use the "iolib" package's exposed functionality.
sm/src/iolib/int_include/ Header files for the "iolib" package that are purely for internal use, and aren't needed by those trying to use the package's exposed functionality.
sm/src/sm/... The directory tree that was to contain all of the code for the remained of the Storage Manager implementation. You can ignore the content of this directory and structure it however you want.

The API

There are two useful interfaces into the IOLIB, depending on whether or not you want to deal with an asynchronous interface.

PageDir

If you want just a simple synchronous interface where you have to be cautious about what kinds of concurrency you cause, use the class tar-directory/sm/src/iolib/int_include/PageDir.H. The file is in the int_include directory because I don't presently consider it to be part of the exposed API from the iolib package. However, to make it exposed, here are the steps:

Move it from the tar-directory/sm/src/iolib/int_include/ directory to the tar-directory/sm/src/iolib/include/ directory.
Modify all of the #include commands in the project's .H / .C files that expect the file to be in its old directory, to instead get it in the new directory.
Add a symbolic link to the file, in the directory tar-directory/sm/include/.

To see how to use the PageDir class, read its .H file and examine the source code of its unit test program.

AsynchPageDir

This class adds several bits of functionality on top of PageDir, which I thought would be well suited to supporting the buffer cache:

Queued commands. This would let there be just one thread in the buffer cache, and it could alternate between formulating/enqueueing new I/O commands and deqeueuing/processing completed I/O commands.
Automatic file growth. If the page files used by the PageDir are getting pretty close to full, it's a good bet they'll need to grow soon. Since growing a page file doesn't necessarily conflict with performing read/wite operations on that file, it's good to predictively kick off a growth operation before the space is needed.

Of course, since I never implemented the pieces of software that would actually use the AsynchPageDir, it remains unproven that the AsynchPageDir's complexity is even needed. Perhaps the PageDir class is all that the rest of the Storage Manager needs for its disk I/O.

To see how to use the AsynchPageDir class, read its .H file and examine the source code of its unit test / benchmark programs.