An Efficient Indexing Technology for Accelerating Data Intensive Science

John Wu, LBNL

FastBit is a software tool for searching large read-only datasets. It organizes user data in a column oriented structure which is efficient for on-line analytical processing (OLAP), and utilizes compressed bitmap indices to further speed up query processing. Analysis has proven the compressed bitmap index to be theoretically optimal for one-attribute queries. Even though there are other optimal indexing methods, bitmap indices are superior because they can be efficiently combined to answer multi-attribute queries whereas other optimal methods can not. In this paper, we first describe the searching capability of FastBit, and then review two applications that make extensive use of FastBit, namely Grid Collector and DEX.