As you may recall, with the release of ERDAS IMAGINE 2010 this past December 2009, each IMAGINE Advantage license provides the customer the capability to batch process up to four simultaneous processes. If you have more IMAGINE Advantage licenses brokered using a floating license manager, each floating IMAGINE Advantage license can add four more simultaneous processes. In this experiment, I tested with one IMAGINE Advantage for 1 – 4 processes and another floating IMAGINE Advantage license for 5 – 8 processes.
With ERDAS IMAGINE 2010 v10.1 the latest version of the ECW JPEG2000 Codec SDK, v4.1 is used. This new version of the SDK has focused in JPEG2000 improvements. This new version of the ECW JPEG2000 Codec SDK will be made available to the market soon.
As discussed in vol 1, thanks to Dell, Inc. for providing the test system and SAM Inc for inspiring the software performance upgrades. The test system is configured as follows:
Dell Precision T7500
Processor: Dual Quad Core, 2.13GHz with 4MB Cache
System RAM: 4GB 1066MHz
Internal Controller: C4 SATA Non-RAID
Internal Disks: Four 1.5TB 7200 RPM SATA with 3.0Gbps, 16MB Data Burst Cache
External Storage: 15 Bay SAS/SATA Array
Array Controller: PERC 6/E SAS RAID, 256MB Memory
Array Disks: Ten 750GB 7200 RPM 3.0Gbps (note above, array can hold 15 disks)
Configuration: RAID 0
Operating System: Microsoft Windows Server 2008 x64, R2
I ran tests using the 2095 uncompressed strip GeoTIFFs of 5000 x 5000 x 3-bands x 8-bit data (each file is 73,282MB). The target compression ratio for both ECW and JPEG2000 was 15:1.
When processing one image at a time, there was a lot of unused processing capacity. For example, when compressing one ECW file at a time 700MB of total RAM was used; 25% of all eight cores were used; and 20% of the disk bandwidth was used.
By using this extra capacity with multiple simultaneous compressions we more effectively use our total available computing power. When compressing eight ECW files simultaneously 1.3GB of total RAM was used; 75% of all eight cores were used; and 90% of the disk bandwidth was used.
While the ERDAS IMAGINE batch engine does not limit its multi-core processing to the number of cores on the system, I stopped at eight because the disk capacity was filling up. I am convinced I could have pushed the number of simultaneous processes to 10 (or more) if I had added five more disks to fill the array container (but these additional five disks were not available).
ERDAS ER Mapper users will have slightly faster times when compressing one image, But, when running a batch of many images and more especially when running simultaneous processes, ERDAS IMAGINE will be significantly faster. In the future, we will incorporate into ERDAS IMAGINE what makes ERDAS ER Mapper faster when processing one image. Below are the times:
| # Processes | ECW Time | JPEG2000 Time |
1 | 2:51:15 | 5:46:44 |
2 | 1:29:27 | 2:44:44 |
3 | 0:43:48 | 1:58:24 |
4 | 0:41:19 | 1:47:03 |
5 | 0:41:37 | 1:06:55 |
6 | 0:35:49 | 0:55:03 |
7 | 0:33:01 | 0:45:51 |
8 | 0:31:53 | 0:43:14 |
Looking at these numbers, we can see that a new day is rising for the remote sensing and GIS user community. Think for a moment, one ECW image was being completed every 0.9 seconds when running eight processes, and one JPEG2000 image was being completed every 1.2 seconds. This means you complete a large compression task in less time than it takes to each you to go eat lunch.
If you need a reason for your boss to spend money on an updated system, in the long term a fast processing configuration like the one I have tested is likely the least expensive productivity enhancement they can buy. All they need to do is determine how fast solutions are needed. Is compressing 2095 images over lunch fast enough?
Volume 1 Post Reference: http://field-guide.blogspot.com/2010/04/single-core-versus-multi-core-image.html
