For many years the remote sensing and photogrammetry community have demanded more performance in processing of image data. (I still remember how excited I was when upgrading from an IBM XP with a 10MB hard disk to a Compaq 386 with a 300MB hard disk.) While, some functions continue to be limited by raw processing performance, most applications can receive massive benefit from a higher performance disk configuration. The next few post will discuss this topic.
Thanks to Dell, Inc. for the test system they provided. This is a cost-effective, yet solid performance server configuration. The test system is as follows:
Dell Precision T7500
Processor: Dual Quad Core, 2.13GHz with 4MB Cache
System RAM: 4GB 1066MHz
Internal Controller: C4 SATA Non-RAID
Internal Disks: Four 1.5TB 7200 RPM SATA with 3.0Gbps, 16MB Data Burst Cache
External Storage: 15 Bay SAS/SATA Array
Array Controller: PERC 6/E SAS RAID, 256MB Memory
Array Disks: Ten 750GB 7200 RPM 3.0Gbps
Configuration: RAID 0
Operating System: Microsoft Windows Server 2008 x64, R2
Since some people will read this blog months after these posts, I will not post prices. Notwithstanding my reluctance to mention prices, I will say that the performance boost will pay for itself. I suggest you look for the price this system on Dell’s website.
This effort actually began a few years ago over the Christmas Holidays when Chuck Patterson (SAM Inc. IT), Dell, Haiyan Qu (ERDAS Customer Support) and I got together to understand the performance of Dell’s EqualLogic Storage Array Network (SAN) device running ERDAS IMAGINE processes. That discussion was in the middle of ERDAS’ development of multi-core batch processing (former working name BatchX). With the completion of the multi-core batch processing effort at ERDAS, Dell and I restarted the testing with standard Disk Arrays. We will expand to SANs over the next few months.
As a benchmark to the speed of the system, I ran tests using the standard copy command in a .bat command file to determine the read-write speed of the configuration. Here are the speeds recorded on the system using 2095 uncompressed GeoTIFFs of 5000 x 5000 x 3-bands x 8-bit data (each file is 73,282MB):
0:14:26 Disk Array to Disk Array
0:23:01 Internal Disk to Disk Array
0:27:20 Disk Array to Internal Disk
0:40:01 Internal Disk, different disks
0:53:56 Internal Disk, same disks
As expected, the RAID 0 Disk Array really boosts performance. To be noted, RAID 0 is not redundant, and therefore long-term data storage on this configuration is not a good idea. But, processing data (especially writing) on a RAID 0 gives you powerful a performance boost.
In the next few posts, I will present file creation speeds of JPEG2000 and ECW in ERDAS IMAGINE 2010.1’s implementation of the ECW JPEG2000 Codec SDK v4.1 using this configuration. I hope you will find posts these useful.
Volume 2 post: http://field-guide.blogspot.com/2010/04/single-core-vs-multi-core-image.html
4 comments:
Hi Paul, thanks for posting. I find this really useful. Here we already use RAID 0 (with 2x up to 4x SATA) in workstations to boost IO performance.
But without the possibility to really test different systems, performance/bottleneck is always a bit of a puzzle to me.
Are you going to make a setup with SSD's as well? I don't expect much of a difference since large images require a good sequential performance, but i guess it depends a bit on how Erdas handles the I/O's when processing.
Anyway, im looking forward to the other benchmarks, thanks!
I am watching Solid State Disks (SSDs). In my opinion, SSDs price per GB is a little high for the performance boost received at this time. There are a few other matters that make SSDs less attractive than hard disk drive arrays.
That being said, I expect the technology to rapidly move into real competition with hard disk drives on laptops and other areas where performance and price is less of an issue than power. Also, SSDs as massive cache buffers to hard disk controllers should give us performance boosts very soon. Can you imagine the performance impact of a 64GB cache on your disk controller / hard disk?
Another area to watch are RAM drives…..
I also have the same computer (only 64-bit and 12GB RAM). The biggest boost of "feeled" improvement for daily work was to install the OS on a Intel 40 GB SSD (http://www.tomshardware.com/news/intel-ssd-value-x25-v-postville,9404.html) and then use the RAID 0 for processing data only. Then a backup of the important data on the RAID 0 array every day after work to another location.
For your setup you run 1 computer against the 15 Bay external array (via 1 GB network?) - do you have any data on connecting 2,4,8 computers to the same 15 Bay and how this will be in comparison?
Actually, the array container is a 15-bay array, but I only have 10 disks loaded and in use. Adding 5 more disks would theoretically speed it up. The tests here were just for serial operating system copies. I will present multi-core processing numbers in the next post (vol 2), a few days from today.
I like the idea of having the OS on a SSD. I bet having ERDAS IMAGINE on a SSD would launch much faster too.
Post a Comment