Thursday, August 11, 2011

The True Storage Cost of Uncompressed Imagery

In the past, we have used 0-level (full resolution) image data and compared its file size to compressed files to determine disk space savings. This approach works when comparing uncompressed data to JPEG, LZW and other non-wavelet compressed images. This approach does not work when comparing full resolution image data to wavelet compressed (e.g. ECW) data formats.

To get good display performance when using large images, reduced resolution datasets (also known as RRDs, OVR, subsample images, pyramid layers) must be pre-calculated and stored with the full resolution data. There are several RRD storage methods, but the most popular is a progressive 4x reduction from the full resolution layer down through each RRD layer, usually down to a size close to 64x64 pixels. The 4x reduction scheme will deliver RRDs which adds 33% to the disk storage requirement already needed by the full resolution data.

Wavelet based formats such as ECW natively contain a full suite of internal RRDs within its file structure. The correct way to compare disk space savings between non-wavelet image formats and wavelet image formats is to compare the non-wavelet image plus its required RRDs disk storage requirement to the wavelet data disk storage requirement.

For example, take an 750GB of image and add 33% for the needed RRDs (assuming using a 4x scheme); the disk space needed for good performance and daily use is 998GB. That’s a lot of space.

If I achieve a 25:1 ECW compression, my ECW file requires 4% of the disk space needed for the full resolution data, and 3% of the total disk space needed (full resolution plus RRDs). The 998GB needed for full resolution imagery plus its RRDS shrank down to 30GB when ECW compressed.

If I achieve a 15:1 ECW compression, my ECW file requires 6.7% f the disk space needed for the full resolution data, and 5% of the total disk space needed for full resolution plus RRDs.

That's a lot of disk space recovered with a just few minutes of compression. Plus, the display time of the ECW is often faster than the display of the uncompressed data. Sometimes much faster.