Thursday, August 11, 2011

The True Storage Cost of Uncompressed Imagery

In the past, we have used 0-level (full resolution) image data and compared its file size to compressed files to determine disk space savings. This approach works when comparing uncompressed data to JPEG, LZW and other non-wavelet compressed images. This approach does not work when comparing 0-level image data to wavelet compression (e.g. ECW) formats.

To get display performance from your large images, reduced resolution datasets (i.e. RRDs, OVR, subsample images, pyramid layers) must be pre-calculated and stored with the 0-level data. There are several RRD storage methods, but the most popular is a progressive 4x reduction from the 0-level data down through each RRD layer. The 4x reduction scheme will deliver RRDs adding 33% to the overall disk storage requirement.

Wavelet based formats natively contain a full suite of internal RRDs within its file structure. The correct way to compare disk space savings between non-wavelet image formats and wavelet formats is to compare the non-wavelet image plus its required RRDs to the wavelet data.

Take 750GB of imagery and add 33% for its needed RRDs (assuming using a 4x scheme). The disk space needed for performance and daily use is 998GB. That’s a lot of space.

If I achieve a 25:1 ECW compression, my ECW file is 4% of the size of the 0-level data, and 3% of the total disk space needed (0-level image plus RRDs). Thus, the 750GB of image data occupying 998GB of disk space shrank down to 30GB.

If I achieve a 15:1 ECW compression, my ECW file is 6.7% of the size of the 0-level data, and 5% of the total disk space needed.

That's a lot of disk space recovered.

No comments: