In the old days, data was stored on punched cards. A file was a deck of cards. Running a job required loading one or more decks of cards in card readers. After some programs ran, the input deck was thrown away and replaced by an output deck. Computer installations used truckloads of cards and had rows and rows of filing cabinets.
The disk drive made cards obsolete because disk drives have many advantages over cards. The computer can read or write to a disk faster than it can read or punch a card. Data occupies less storage on disk than on a card. Record lengths can be greater than 80 or 96 bytes. Files can be processed in random order. You can probably think of more.
If I were to tell you that theres something that compares to disk drives the way disk drives compare to paper cards, would you get excited? Well, such an animal does exist. Its called memory.
Memory has advantages over disk drives. For starters, it occupies less physical space. Because its not mechanical, memory is less prone to failure than disk drives are, and the computer can access memory much more quickly. Now that memory is getting cheaper and cheaper, people can afford to have more of it in their computers. People who understand these things better than I do tell me the price of memory is going to continue to drop.
Store It in Memory
If youre not familiar with the acronym IMDS, you should get to know it. IMDS stands for in-memory database system, and it may soon be as much a part of your daily life as DBMS (database management system) is. An IMDS is a database management system that stores data in memory, rather than on disk. IMDSs offer the following advantages over disk- based DBMSs:
Faster access. Retrieving a row from a table in memory is much faster than retrieving a row from disk.
Elimination of the middle man. An IMDS accesses memory directly. In a disk- based system, the data is copied from disk into buffers, which are copied into program variables.
Less overhead. IMDS software requires less memory to run than a DBMS engine does.
Using an IMDS is not much different from using a DBMS. Access is through SQL, either through direct calls to the IMDS software or through an ODBC driver.
Its Not Vaporware
I am aware of three commercially available IMDSs.
Angara Database Systems (www.angara.com) markets the Angara Data Server, which, it claims, is 20 times faster than the cache of disk-based database management systems.
TimesTen Performance Softwares (www.timesten.com) IMDS product, which is also called TimesTen, is supposed to be able to process thousands of database operations per second. TimesTen also claims that its engine uses one-tenth the CPU instructions of a DBMS.
DataBlitz, from Lucent Technologies, grew from research at Bell Labs. To learn all you ever wanted to know about IMDSs, point your Web browser to www.lucent.com and search the site for DataBlitz. Youll find several white papers there, as well as other information.
Dont throw those disk drives away yet. While the existence of IMDSs is wonderful and exciting, the DBMS is not yet obsolete. For one thing, computers dont yet have enough memory to contain most databases. Perhaps someday well be able to buy a PC with a terabyte of memory, but that day wont be anytime soon.
IMDSs also have their limitations. For example, a TimesTen database cannot be larger than 2 GB. I expect to see such limitations relaxed in the near future.
Theres also the question of reliability. What happens to the database when the computer goes down? The IMDS packages use disk as backup. As the database is modified, the transactions are logged to disk. In addition, some IMDSs also make snapshots of the database from time to time and copy those to disk. IMDS vendors claim that their systems are as reliable as disk-based systems.
For now, IMDS vendors are targeting their products toward applications that require speed in real time, such as searching a database. For more traditional database applications, such as printing invoices or payroll checks, a DBMS is still the best way to go. Making an IMDS and a DBMS work with one another is not difficult to do, since the IMDS uses the same interface standards that relational DBMSs use.
The Future Looks Good
If you wonder about the future of the information processing industry, as I often do, maybe you share my enthusiasm for in-memory database systems. If so, be sure to read Hector Garcia-Molinas article, Fastest in Memory, in the August 24, 1999, issue of Intelligent Enterprise.
In the late 1980s, one of my former bosses used to predict that memory would someday replace disk drives. I didnt doubt he was right. I just didnt think it would happen so soon.