Publication

Column-oriented database systems have shown to perform better than traditional row-oriented database systems on analytical workloads found in decision support and business intelligence applications. Moreover, lightweight compression schemes have shown to significantly improve the query processing performance in these systems. One such a lightweight compression scheme is to use a dictionary in order to replace long (variable-length) values of a certain domain with shorter (fixed-size) integer codes. In order to further improve expensive query operations such as sorting and searching, column-stores often use order-preserving encoding schemes for a dictionary. In contrast to the existing work, we argue that a dictionary-based order-preserving compression scheme does not only pay-off for attributes with a small fixed domain size but also for long string attributes with a large domain size which might change over time. Consequently, in this paper we introduce new data structures that efficiently support a dictionary-based order-preserving compression for (variable-length) string attributes with a large domain size that is likely to change over time. The main idea is that we model a dictionary as a table that specifies a mapping from string-values to arbitrary integer codes (and vice versa). Moreover, we introduce a new indexing approach that provides efficient access paths to such a dictionary while compressing the index data. Our experiments show that our data structures are as fast as (or in some cases even faster than) other state-of-the-art data structures for dictionaries while being less memory intensive.

@inproceedings{abc,
	abstract = {Column-oriented database systems have shown to perform better than traditional row-oriented database systems on analytical workloads found in decision support and business intelligence applications. Moreover, lightweight compression schemes have shown to significantly improve the query processing performance in these systems. One such a lightweight compression scheme is to use a dictionary in order to replace long (variable-length) values of a certain domain with shorter (fixed-size) integer codes. In order to further improve expensive query operations such as sorting and searching, column-stores often use order-preserving encoding schemes for a dictionary.

In contrast to the existing work, we argue that a dictionary-based order-preserving compression scheme does not only pay-off for attributes with a small fixed domain size but also for long string attributes with a large domain size which might change over time. Consequently, in this paper we introduce new data structures that efficiently support a dictionary-based order-preserving compression for (variable-length) string attributes with a large domain size that is likely to change over time.
The main idea is that we model a dictionary as a table that specifies a mapping from string-values to arbitrary integer codes (and vice versa). Moreover, we introduce a new indexing approach that provides efficient access paths to such a dictionary while compressing the index data.
Our experiments show that our data structures are as fast as (or in some cases even faster than) other state-of-the-art data structures for dictionaries while being less memory intensive.},
	author = {Carsten Binnig and Stefan Hildenbrand and Franz F{\"a}rber},
	booktitle = {Research},
	title = {Dictionary-based Order-preserving String Compression for Main Memory Column Stores},
	url = {http://doi.acm.org/10.1145/1559845.1559877},
	year = {2009}
}