With the release of ArcGIS 10, ESRI’s desktop products support ECW image files natively through GDAL. The GDAL move was part of ESRI's switch from the ERDAS Raster Data Objects (RDO) that ESRI paid ERDAS to use. It was very low cost, as ESRI has historically been a business partner, but ESRI wanted to go in another direction. (The ESRI community can expect a bumpy raster ride for a few releases.)
The GDAL version used in ArcGIS 10 uses an older version of the ERDAS ECW JPEG2000 SDK (ECW SDK), so opacity is not supported and the speed is a little slower than what will be available using the latest version of the ECW SDK, v4.1.
ERDAS is working with GDAL to upgrade GDAL to the latest version of the ECW SDK, scheduled to be complete before Feb 1, 2011. Once the GDAL work is complete, software companies using GDAL (such as ESRI) need only download the Read Only ECW SDK and build in their desktop application as defined by GDAL and according to the ECW SDK EULA for free ECW, ECWP and JPEG2000 desktop read capability.
GDAL is a SDK and the ECW SDK EULA states a SDK cannot deliver the ECW SDK with their SDK. The solution is for GDAL to build against the ECW SDK, but not deliver the ECW SDK. Then the companies using GDAL can simply download the Read Only ECW SDK from ERDAS and put GDAL and the ECW SDK in their desktop software application.
The beauty of the way the Read Only and Full ECW SDKs were put together, once the developer has done the work for ECW, the work to implement ECWP and JPEG2000 may take as few as 30 more minutes’ development time.
If a company wishes their software application to write ECW and JPEG2000 data, they can use the same GDAL approach noted above; and with a purchase of the Full ECW SDK from ERDAS, they add writing ECW and JPEG2000 data.
For ECW support in ArcGIS Server see: http://field-guide.blogspot.com/2011/04/news-release-erdas-releases-new-product.html
For streaming ECW and JP2 data via ECWP in ArcGIS desktop see: http://field-guide.blogspot.com/2011/05/esri-adding-for-ecwp-in-arcgis.html
Why ECW?
Many in the geospatial community correctly think of ECW (Enhanced Compression Wavelet) as a data format that saves disk space. A more powerful aspect of ECW, because of its fundamental mathematical breakthrough enabling fast compression, the same breakthrough provides very fast decompression as well; while only using a small amount of RAM. Whether accessing ECW on a desktop, over a LAN, or over the internet, ECW is very fast. In many cases, ECW is much faster than accessing the imagery in an uncompressed format.
What is ECWP?
Geospatial images can be hundreds or thousands of gigabytes in size. Traditional mechanisms for serving image data over the internet are inadequate when high-speed performance is required. ECWP (Enhanced Compression Wavelet Protocol) leverages the benefits of a the mathematically efficient ECW format and allows the server to stream these large geospatial images to a user's application, rather than sending a regular image ver HTTP, while only using a tiny amount of RAM. JPEG2000 can be streamed with ECWP as well, with higher performance than JPIP.
Why JPEG2000?
JPEG2000 is an ISO-certified wavelet compression image format that is commonly used for geospatial imagery. The format is defined by the Joint Photographic Experts Group. Because JPEG2000 was designed by a committee to do a great many things, it will never be as fast as ECW. Because JPEG2000 compression and decompression technology is similar to ECW, it was simple to add this capability to the ERDAS ECW JPEG2000 SDK, even supporting JPEG2000 streaming within ECWP. But, because JPEG2000 was created by a committee to a great many things, JPEG2000 is unlikely to ever be as fast as ECW.
ECW on Wikipedia
Tuesday, September 21, 2010
Monday, September 20, 2010
ERDAS Releases New Version of ERDAS ECW JPEG2000 SDK
As many of you know, I am also the Product Manager for the ERDAS ECW JPEG2000 SDK. I am proud of the work the SDK development team in Perth has done. Components of this released version of the SDK are being used in all ERDAS 2010 v10.1 products. There is a discussion forum for the SDK on ERDAS Communities.
The following was released to the press earlier today:
Norcross, GA — ERDAS announces the release of a new version of the ERDAS ECW JPEG2000 SDK, which enables software developers to include robust support for selected wavelet compression formats and protocols in the desktop applications they create.
The ERDAS ECW JPEG2000 SDK provides read and write support for the visually-lossless ECW and numerically-lossless JPEG2000 imagery formats, and read support for the ECWP protocol. Enhanced Compression Wavelet (ECW) is a compressed format designed specifically for geospatial imagery, which can quickly compress and decompress huge images using minimal memory. Enhanced Compression Wavelet Protocol ECWP) allows streaming of ECW images, enabling rapid delivery of terabyte-sized images over the internet using inexpensive server technology. JPEG2000 is an ISO-certified compressed image format that is commonly used for geospatial imagery.
Originally used in the ER Mapper products, this SDK has been implemented in all ERDAS software. ERDAS IMAGINE, LPS and ERDAS APOLLO use the SDK to provide support for these wavelet compression and transmission technologies. For over ten years, this industry-proven SDK has produced stable, high-quality commercial applications.
“The ERDAS ECW JPEG2000 SDK allows any software developer to quickly implement industry-proven support of the ECW and JPEG2000 formats in their desktop applications, easily increasing the value and appeal of those applications to end users. On top of that, we’ve included changes in the latest release that result in remarkable performance increases, allowing your customers to decode ECW and JPEG2000 images and stream via ECWP faster than ever before,” said Mark Sheridan, Director of Engineering at ERDAS and one of the original developers of ER Mapper and ECW technology.
Developers can use the newest SDK to implement read and write support for ECW and JPEG200 images with opacity bands, allowing images to overlay other imagery cleanly without showing compression artifacts around the edges. Also, the SDK can now store compressed blocks delivered over an ECWP stream in the client cache, so previously viewed data can be opened faster. This also reduces the server’s bandwidth requirement, since less data is transferred to the client.
The C++ API provided in the SDK includes new functions for file decoding, rendering and accessing lower-level JPEG2000 features, such as the number of quality layers to include during decoding. Additionally, the API allows developers to retrieve and set options related to opacity, fussy or resilient modes in the decoder, the configuration of J2I index files, and caching parameters.
ERDAS distributes a free version of the SDK that enables read support for ECW, JPEG2000 and ECWP on its product web site.
For more information about ERDAS or its products and services, please call +1 770 776 3400, toll free +1 877 GO ERDAS, or visit www.erdas.com.
The following was released to the press earlier today:
Norcross, GA — ERDAS announces the release of a new version of the ERDAS ECW JPEG2000 SDK, which enables software developers to include robust support for selected wavelet compression formats and protocols in the desktop applications they create.
The ERDAS ECW JPEG2000 SDK provides read and write support for the visually-lossless ECW and numerically-lossless JPEG2000 imagery formats, and read support for the ECWP protocol. Enhanced Compression Wavelet (ECW) is a compressed format designed specifically for geospatial imagery, which can quickly compress and decompress huge images using minimal memory. Enhanced Compression Wavelet Protocol ECWP) allows streaming of ECW images, enabling rapid delivery of terabyte-sized images over the internet using inexpensive server technology. JPEG2000 is an ISO-certified compressed image format that is commonly used for geospatial imagery.
Originally used in the ER Mapper products, this SDK has been implemented in all ERDAS software. ERDAS IMAGINE, LPS and ERDAS APOLLO use the SDK to provide support for these wavelet compression and transmission technologies. For over ten years, this industry-proven SDK has produced stable, high-quality commercial applications.
“The ERDAS ECW JPEG2000 SDK allows any software developer to quickly implement industry-proven support of the ECW and JPEG2000 formats in their desktop applications, easily increasing the value and appeal of those applications to end users. On top of that, we’ve included changes in the latest release that result in remarkable performance increases, allowing your customers to decode ECW and JPEG2000 images and stream via ECWP faster than ever before,” said Mark Sheridan, Director of Engineering at ERDAS and one of the original developers of ER Mapper and ECW technology.
Developers can use the newest SDK to implement read and write support for ECW and JPEG200 images with opacity bands, allowing images to overlay other imagery cleanly without showing compression artifacts around the edges. Also, the SDK can now store compressed blocks delivered over an ECWP stream in the client cache, so previously viewed data can be opened faster. This also reduces the server’s bandwidth requirement, since less data is transferred to the client.
The C++ API provided in the SDK includes new functions for file decoding, rendering and accessing lower-level JPEG2000 features, such as the number of quality layers to include during decoding. Additionally, the API allows developers to retrieve and set options related to opacity, fussy or resilient modes in the decoder, the configuration of J2I index files, and caching parameters.
ERDAS distributes a free version of the SDK that enables read support for ECW, JPEG2000 and ECWP on its product web site.
For more information about ERDAS or its products and services, please call +1 770 776 3400, toll free +1 877 GO ERDAS, or visit www.erdas.com.
Monday, September 13, 2010
ERDAS IMAGINE 2010 Ribbon UI: Invest one week and experiment for yourself
There have been many positive comments about the ERDAS IMAGINE 2010 Ribbon user interface (UI). There are some users who have not made the transition, saying they do not want to learn a new UI. I understand the comfortable feeling we have with an old UI. I remember not liking the move from a command prompt / menu system of the ERDAS 7.x series and the pre-ArcGIS versions to the graphical user interface (GUI) of ERDAS IMAGINE in the 1990s and of ArcGIS in the 2000s. Notwithstanding humans seeking to stay within a comfort zone of what is known, if we are not learning something new every day, are we wasting the learning skills we developed in all our years in school?
Val Clarke, a wise friend of mine and Dr. John Jensen's, told me in about 1983, "There is no growth without change." I have discovered over the subsequent 27 years, he is correct.
To help with ERDAS' customers transition (or growth), ERDAS wrote a document last year to help the transition. This document is named, "Increasing Workflow Efficiency with ERDAS IMAGINE 2010" and can be found on the ERDAS IMAGINE Product page under the "Product Literature" tab (see: http://www.erdas.com/tabid/84/currentid/1050/default.aspx).
The move to the Ribbon design has allowed ERDAS to expose more of the power of ERDAS IMAGINE than ever before. We spend a lot of time on placement of functions, and continue to debate and improve each development iteration. If you have not spent one-week using the ERDAS IMAGINE 2010 Ribbon User Interface, you are missing out. Invest one-week and then tell me if the old UI is better than the new.
Don't take my word for it. Invest one week and experiment for yourself.
Val Clarke, a wise friend of mine and Dr. John Jensen's, told me in about 1983, "There is no growth without change." I have discovered over the subsequent 27 years, he is correct.
To help with ERDAS' customers transition (or growth), ERDAS wrote a document last year to help the transition. This document is named, "Increasing Workflow Efficiency with ERDAS IMAGINE 2010" and can be found on the ERDAS IMAGINE Product page under the "Product Literature" tab (see: http://www.erdas.com/tabid/84/currentid/1050/default.aspx).
The move to the Ribbon design has allowed ERDAS to expose more of the power of ERDAS IMAGINE than ever before. We spend a lot of time on placement of functions, and continue to debate and improve each development iteration. If you have not spent one-week using the ERDAS IMAGINE 2010 Ribbon User Interface, you are missing out. Invest one-week and then tell me if the old UI is better than the new.
Don't take my word for it. Invest one week and experiment for yourself.
Wednesday, September 1, 2010
Hexagon participates at Intergraph 2010 event
Today Ola Rollén, President and CEO of Hexagon AB, participates at the Intergraph Corporation's annual international users' conference in Las Vegas, USA. The event "Intergraph 2010" hosts approximately 2 000 Intergraph customers from across the world and offers previews of new Intergraph technology, presentations by industry experts and hands-on training of Intergraph technology.
In a keynote address Ola Rollén comments on the planned acquisition of Intergraph announced on 7 July 2010. He reiterates Hexagon's intentions of continuous commitment and investment in Intergraph's vision, solutions, customers and employees. Ola Rollén also confirms Hexagon's intent to support Intergraph's product roadmap and to further invest in research and development.
"It is our intention that Intergraph's solutions will become Hexagon's core software platform, providing differentiated and vertically-focused software solutions. By combining Intergraph's technologies with our global resources and technologies, Hexagon will be able to create new exciting solutions to customers going forward", says Ola Rollén.
The acquisition of Intergraph is subject to completion of regulatory process and satisfaction of customary closing conditions. Competition law notifications have been submitted to the relevant regulatory authorities and the applicable waiting periods have expired. Completion of the remaining regulatory procedures is pending. Financial consolidation is estimated to take place in the fourth quarter of 2010.
At closing, Intergraph will become a fully owned subsidiary of Hexagon AB and will operate as a separate Hexagon division under the Intergraph name and branding. Following closing, it is planned that Ola Rollén will assume the role of CEO of Intergraph in accordance with the Hexagon model for successful integration into the Hexagon Group. Following closing, the two Intergraph divisions Process, Power & Marine and Security Government & Infrastructure will continue to operate under the leadership of Gerhard Sallinger and John Graham, respectively.
For further information please contact: Sara Kraft, Corporate Communications, Hexagon AB, +46 8 601 26 23, sara.kraft@hexagon.se
Extracted 09/01/2010 http://investors.hexagon.se/index.php?p=press&afw_lang=en
See Ola's Keynote at the Intergraph User Conference
Monday, July 19, 2010
Understanding the Target Compression Ratio when using the ERDAS ECW JPEG2000 SDK
It is important to note that when using the ERDAS ECW JPEG2000 SDK, the ECW and JPEG2000 target compression ratio makes no guarantees about the actual output size that will be achieved. Images with certain features (for example, air photos showing large regions of a similar color, like background values, oceans or forests) are easier to compress than others (completely random images).
When compressing images there is a tradeoff between the degree of compression achieved and the quality of the resulting image. The highest rates of compression can only be achieved by discarding some less important data from the image, known as lossy decompression. In the ERDAS ECW JPEG2000 SDK, the target compression ratio is an abstract figure representing your choice in this tradeoff. It approximates the likely ratio of input file size to output file size, given certain parameters for compression.
It is often unwise to try and force each image into a compressed file of the same size. This is because the resulting compressed images will have widely varying quality levels which may reduce their usefulness and any subsequent processing will have decreased value when combining different image tiles of different quality compression together.
Below is a table taken from the ERDAS IMAGINE 2010 Help (to be released later in 2011). This table was created from in-house testing and customer feedback.
When compressing images there is a tradeoff between the degree of compression achieved and the quality of the resulting image. The highest rates of compression can only be achieved by discarding some less important data from the image, known as lossy decompression. In the ERDAS ECW JPEG2000 SDK, the target compression ratio is an abstract figure representing your choice in this tradeoff. It approximates the likely ratio of input file size to output file size, given certain parameters for compression.
It is often unwise to try and force each image into a compressed file of the same size. This is because the resulting compressed images will have widely varying quality levels which may reduce their usefulness and any subsequent processing will have decreased value when combining different image tiles of different quality compression together.
Below is a table taken from the ERDAS IMAGINE 2010 Help (to be released later in 2011). This table was created from in-house testing and customer feedback.
| Imagery | Target Compression Ratio | |
| Visually Lossless RGB Image | Crisp Image Interpretation (2x zoom) | 4:1 |
| Near Visually Lossless RGB Image | Clear Image Interpretation (2x zoom) | 6:1 |
| RGB Image | High Quality Printed Maps and typical GIS applications | 15:1 to 25:1 |
| RGB Image | Internet or Email Distribution | 15:1 to 40:1 |
| Visually Lossless Panchromatic Image | Crisp Image Interpretation (2x zoom) | 2:1 |
| Near Visually Lossless Panchromatic Image | Clear Image Interpretation (2x zoom) | 3:1 |
| Panchromatic Image | High Quality Printed Maps and typical GIS applications | 10:1 to 15:1 |
| Panchromatic Image | Internet or Email Distribution | 15:1 to 30:1 |
| Visually Lossless Multispectral Image | Crisp Image Interpretation (2x zoom) | 3:1 |
| Near Visually Lossless Multispectral Image | Clear Image Interpretation (2x zoom) | 4:1 |
| Numerically Lossless RGB, Panchromatic, and Multispectral (JPEG 2000 Only) | Imagery with perfect numeric reconstruction | 1:1 |
Thursday, July 8, 2010
Numerically Lossless or Visually Lossless Wavelet Compression
I hear so very often, we need lossless image compression (meaning numerically lossless - NL). I respect the point much more when I hear it from remote sensing and photogrammetry scientists than from GIS users.
Why my differentiation? Am I diss’ing the GIS folks? I hope not, that is not my intention.
Way back in 2001 I did a little test. I was at Georgia Tech Center for GIS (CGIS) and was asked by the State of Georgia to determine what compression level was acceptable to compress the state’s Color-Infrared aerials. Wavelet NL was not available through COTS software at the time (MrSID and ECW) and CGIS was not funded to research a new compression method. We were told to research what COTS compression level to use.
So, we pulled together engineering students, business students, architectural students, geospatial scientists, and secretaries. Most had some geospatial training and a few did not. We displayed an 8-bit uncompressed image, and compressed versions of the same image at 10:1, 15:1, 20:1, 25:1 and 30:1 compressions.
The people were allowed display images side-by-side, use swipe, blend, and fade functions. They could zoom in and out, but no further than a 4x zoom.
Only one person could see the compression artifacts at a 1:1 zoom before 20:1 compression. At 20:1 all the experienced remote sensing people could see a few artifacts. The experienced GIS people could see the artifacts at a 2x zoom of 15:1, but only one of non-geospatial people (a business student) could see the artifacts at 2x 15:1. (Later she decided to work as a research assistant for me and is still in the geospatial industry).
My point is… many in the geospatial industry push the 2 – 2.5x file size saving when using NL compression; when many trained geospatial people cannot see a compression artifacts in 3-band image compressions below 10:1 unless they zoom in to 4x. (Sure, we all know of the exceptions; the remote sensing, and photogrammetry experts duly noted above; but these are the exception, not the rule.)
So why not compress at a VL level when you are not doing precise remote sensing or photogrammetry? The medical imaging industry is gravitating on between 8:1 and12:1. Is what we are doing in the geospatial world so important we have to preserve more precision than the medical imaging? Are we protecting more precision than our accuracy supports?
I for one think we are wasting space and time when we do not compress to at least the VL compression level.
A final note, I am running tests to know at what level we can wavelet compress data and not affect autocorrelation, classification, and vegetation indices. Have any of you dared such tests? Care to let us know?
Why my differentiation? Am I diss’ing the GIS folks? I hope not, that is not my intention.
Way back in 2001 I did a little test. I was at Georgia Tech Center for GIS (CGIS) and was asked by the State of Georgia to determine what compression level was acceptable to compress the state’s Color-Infrared aerials. Wavelet NL was not available through COTS software at the time (MrSID and ECW) and CGIS was not funded to research a new compression method. We were told to research what COTS compression level to use.
So, we pulled together engineering students, business students, architectural students, geospatial scientists, and secretaries. Most had some geospatial training and a few did not. We displayed an 8-bit uncompressed image, and compressed versions of the same image at 10:1, 15:1, 20:1, 25:1 and 30:1 compressions.
The people were allowed display images side-by-side, use swipe, blend, and fade functions. They could zoom in and out, but no further than a 4x zoom.
Only one person could see the compression artifacts at a 1:1 zoom before 20:1 compression. At 20:1 all the experienced remote sensing people could see a few artifacts. The experienced GIS people could see the artifacts at a 2x zoom of 15:1, but only one of non-geospatial people (a business student) could see the artifacts at 2x 15:1. (Later she decided to work as a research assistant for me and is still in the geospatial industry).
My point is… many in the geospatial industry push the 2 – 2.5x file size saving when using NL compression; when many trained geospatial people cannot see a compression artifacts in 3-band image compressions below 10:1 unless they zoom in to 4x. (Sure, we all know of the exceptions; the remote sensing, and photogrammetry experts duly noted above; but these are the exception, not the rule.)
So why not compress at a VL level when you are not doing precise remote sensing or photogrammetry? The medical imaging industry is gravitating on between 8:1 and12:1. Is what we are doing in the geospatial world so important we have to preserve more precision than the medical imaging? Are we protecting more precision than our accuracy supports?
I for one think we are wasting space and time when we do not compress to at least the VL compression level.
A final note, I am running tests to know at what level we can wavelet compress data and not affect autocorrelation, classification, and vegetation indices. Have any of you dared such tests? Care to let us know?
Wednesday, July 7, 2010
Hexagon Aquires Intergraph
Hexagon, the parent company of Leica Geosystems (the parent company of ERDAS) announced today it acquired Intergraph yesterday, July 6, 2010. With this acquisition, Hexagon owns the two most respected areal imaging / LIDAR sensor companies. This capability along with the high precision GPS, CAD, GIS and Remote Sensing software capabilities, and different market access should make for interesting synergies among Hexagon's geospatial companies over the next few years.
Hexagon Announcement
Intergraph Announcement
With this acquisition, has Ola Rollen, CEO of Hexagon positioned Hexagon as the largest geospatial organization on the planet? Looking at the Intergraph, Leica Geosystems, NovAtel and ERDAS numbers, the case can surely be proposed.
Ola has told his ERDAS and Leica Geosystems people Hexagon is dedicated to the software business. This move puts real teeth behind his comments and right in the middle of the geospatial industry.
Ola Rollen's Press Conference
Hexagon will move Intergraph's sensor unit over to Leica Geosystems and ERDAS, Inc. over to Intergraph. This will consolidate geospatial hardware under Leica Geosystems and geospatial software under Intergraph.
Stay Tuned!
Hexagon Announcement
Intergraph Announcement
With this acquisition, has Ola Rollen, CEO of Hexagon positioned Hexagon as the largest geospatial organization on the planet? Looking at the Intergraph, Leica Geosystems, NovAtel and ERDAS numbers, the case can surely be proposed.
Ola has told his ERDAS and Leica Geosystems people Hexagon is dedicated to the software business. This move puts real teeth behind his comments and right in the middle of the geospatial industry.
Ola Rollen's Press Conference
Hexagon will move Intergraph's sensor unit over to Leica Geosystems and ERDAS, Inc. over to Intergraph. This will consolidate geospatial hardware under Leica Geosystems and geospatial software under Intergraph.
Stay Tuned!
Wednesday, June 16, 2010
The first ERDAS Manual
Brad Skelton, as a hobby over the years, has been the ERDAS archivist. I enjoy visiting his office and looking at the more than 30 years of ERDAS historical ‘stuff.’ One bit of ‘stuff’ Brad has archived carries a lot of significance, the very first ERDAS Manual. I am embarking on an effort to scan and OCR the document into a searchable PDF that looks exactly like the original document. Two beautiful and multi-talented ERDAS ladies, Candy Billips and Jenn Gazdziak, will help me with the effort. (Truth be told, I could not do it without their help.)
The first ERDAS Manual was written by Mark Finlay, Larry Brantley and Brad Skelton. Andrea Gernazian (Bruce Rado's, one of the three ERDAS founders, wife) proof-read the manual and performed word processing duties along with Melissa Ergle using Wordstar on a Cromemco Z80 microcomputer. The entire document was printed with great effort on a Brother daisy wheel printer (which had the tendency to jam every few pages, or smear printer dust over the pages). Andrea coined the phrase “The Oh, Brother! Printer”.
I quoted from another page of the first ERDAS Manual in the blog post, A Brief History of ERDAS IMAGINE
Below is the introduction page (page 2-1) from the first manual. Note the spelling error ’digitial,’ which everyone missed in 1983.
INTRODUCTION Rev 7.1 / 1 June 1983
A. ERDAS 2300/2400 SYSTEM
I. Introduction
The ERDAS 2300/2400 system is a complete turn key stand alone image processing system for performing image analysis of remotely sensed data, such as LANDSAT or SPOT satellite data, video digitized image data such as aerial photographs, or gridded polygon data such as digitized soils or topography maps. The ERDAS software package that comes with the 2400 system provides many standard image processing functions , a complete and integrated geographic information system (GIS), polygon capture software for use with a table digitizer, and many data base management utilities, all run in a menu-driven and highly interactive environment. The ERDAS 2400 system may be configured to work with several different image processors, ranging from the low cost Digital Graphics CAT system to the more powerful, higher resolution displays such as the Raster Technologies and Gould (DeAnza) image processors. Through the GIS software, many different types of data may be combined in one analysis to provide solutions to problems that cannot be solved using only one type of data. The ERDAS mapping package can produce true scale hard copy map output on the graphics line printer provided as standard equipment. The amount of data that can be processed and stored on the system is limited only by the amount of disk space that is purchased with the system, and more disk space can always be added later if the need arises. As little as 20 megabytes of storage can be used for small scale image processing requirements, or as much as 1000 megabytes can be purchased. A standard 9-track tape drive is also provided with the system to allow for easy access to digital image data from a large number of sources. A number of different CPUs, ranging from the PDP 11/23+, and MICRO-VAX for small scale operations, up to the VAX 11/780, for large scale image processing, are available to satisfy the most demanding applications. An optional table digitizer may also be purchased to allow the system to capture data directly from maps of almost any scale. Multi-user capability is also provided with the ERDAS system. One or many users may perform data processing simultaneously, and in fact, each individual user can run and manage several concurrently running programs so that tasks such as loading a tape can be performed in the background, while other, more interactive tasks, can be done at the console. Data files may be shared by several users or protected from unauthorized access by the file protection system of the RSXll-M or VMS operating system. The ERDAS system is designed to provide a powerful software tool for managing, displaying, and processing digitial image and geographic data, without requiring sophisticated knowledge of computers or programming.
Copyright 1983, 1984 by ERDAS, Inc. All rights reserved
The first ERDAS Manual was written by Mark Finlay, Larry Brantley and Brad Skelton. Andrea Gernazian (Bruce Rado's, one of the three ERDAS founders, wife) proof-read the manual and performed word processing duties along with Melissa Ergle using Wordstar on a Cromemco Z80 microcomputer. The entire document was printed with great effort on a Brother daisy wheel printer (which had the tendency to jam every few pages, or smear printer dust over the pages). Andrea coined the phrase “The Oh, Brother! Printer”.
I quoted from another page of the first ERDAS Manual in the blog post, A Brief History of ERDAS IMAGINE
Below is the introduction page (page 2-1) from the first manual. Note the spelling error ’digitial,’ which everyone missed in 1983.
INTRODUCTION Rev 7.1 / 1 June 1983
A. ERDAS 2300/2400 SYSTEM
I. Introduction
The ERDAS 2300/2400 system is a complete turn key stand alone image processing system for performing image analysis of remotely sensed data, such as LANDSAT or SPOT satellite data, video digitized image data such as aerial photographs, or gridded polygon data such as digitized soils or topography maps. The ERDAS software package that comes with the 2400 system provides many standard image processing functions , a complete and integrated geographic information system (GIS), polygon capture software for use with a table digitizer, and many data base management utilities, all run in a menu-driven and highly interactive environment. The ERDAS 2400 system may be configured to work with several different image processors, ranging from the low cost Digital Graphics CAT system to the more powerful, higher resolution displays such as the Raster Technologies and Gould (DeAnza) image processors. Through the GIS software, many different types of data may be combined in one analysis to provide solutions to problems that cannot be solved using only one type of data. The ERDAS mapping package can produce true scale hard copy map output on the graphics line printer provided as standard equipment. The amount of data that can be processed and stored on the system is limited only by the amount of disk space that is purchased with the system, and more disk space can always be added later if the need arises. As little as 20 megabytes of storage can be used for small scale image processing requirements, or as much as 1000 megabytes can be purchased. A standard 9-track tape drive is also provided with the system to allow for easy access to digital image data from a large number of sources. A number of different CPUs, ranging from the PDP 11/23+, and MICRO-VAX for small scale operations, up to the VAX 11/780, for large scale image processing, are available to satisfy the most demanding applications. An optional table digitizer may also be purchased to allow the system to capture data directly from maps of almost any scale. Multi-user capability is also provided with the ERDAS system. One or many users may perform data processing simultaneously, and in fact, each individual user can run and manage several concurrently running programs so that tasks such as loading a tape can be performed in the background, while other, more interactive tasks, can be done at the console. Data files may be shared by several users or protected from unauthorized access by the file protection system of the RSXll-M or VMS operating system. The ERDAS system is designed to provide a powerful software tool for managing, displaying, and processing digitial image and geographic data, without requiring sophisticated knowledge of computers or programming.
Copyright 1983, 1984 by ERDAS, Inc. All rights reserved
Thursday, June 3, 2010
Utah State University Image Standardization Tools
Years ago I ran into a tool built by the Remote Sensing/GIS lab and Utah State University. I recently stumbled onto the tool once again and immediately thought of sharing the tool with you folks.
The website states, “This website provides three tools that create ERDAS Imagine TM spatial models (.gmd format). Each tool creates a different .gmd file providing a slightly different approach to standardization. These tools are designed for Landsat 5 TM and Landsat 7 ETM+ scenes and require, as input, the header file (*.h1) accompanying NLAPS formatted imagery from the USGS at Eros Data Center.”
It offers,
- DN-to-Reflectance Conversion
- COST Atmospheric Correction
- COST without Tau (Dark Object Subtraction) Atmospheric Correction
Check it out and have fun...
Thursday, May 6, 2010
Condolences to Dr. Robert Moses' family, friends, and colleagues at PCI Geomatics
I send out my condolences to Dr. Robert Moses' family, friends, and colleagues at PCI Geomatics. Dr. Moses passed away Monday, May 3rd at his home in Chelsea, Quebec, Canada.
PCI Press Release: http://www.pcigeomatics.com/pressnews/Dr_Robert_Moses.pdf
PCI Press Release: http://www.pcigeomatics.com/pressnews/Dr_Robert_Moses.pdf
Sunday, April 25, 2010
Single core vs. multi-core image processing (vol 2)
As discussed in Single core vs. multi-core image processing (vol 1), I want to present the times for ECW and JPEG2000 compression in ERDAS IMAGINE 2010 v10.1. I tested using the Export to ECW and Export to JPEG2000 commands, running in batch mode.
As you may recall, with the release of ERDAS IMAGINE 2010 this past December 2009, each IMAGINE Advantage license provides the customer the capability to batch process up to four simultaneous processes. If you have more IMAGINE Advantage licenses brokered using a floating license manager, each floating IMAGINE Advantage license can add four more simultaneous processes. In this experiment, I tested with one IMAGINE Advantage for 1 – 4 processes and another floating IMAGINE Advantage license for 5 – 8 processes.
With ERDAS IMAGINE 2010 v10.1 the latest version of the ECW JPEG2000 Codec SDK, v4.1 is used. This new version of the SDK has focused in JPEG2000 improvements. This new version of the ECW JPEG2000 Codec SDK will be made available to the market soon.
As discussed in vol 1, thanks to Dell, Inc. for providing the test system and SAM Inc for inspiring the software performance upgrades. The test system is configured as follows:
Dell Precision T7500
Processor: Dual Quad Core, 2.13GHz with 4MB Cache
System RAM: 4GB 1066MHz
Internal Controller: C4 SATA Non-RAID
Internal Disks: Four 1.5TB 7200 RPM SATA with 3.0Gbps, 16MB Data Burst Cache
External Storage: 15 Bay SAS/SATA Array
Array Controller: PERC 6/E SAS RAID, 256MB Memory
Array Disks: Ten 750GB 7200 RPM 3.0Gbps (note above, array can hold 15 disks)
Configuration: RAID 0
Operating System: Microsoft Windows Server 2008 x64, R2
I ran tests using the 2095 uncompressed strip GeoTIFFs of 5000 x 5000 x 3-bands x 8-bit data (each file is 73,282MB). The target compression ratio for both ECW and JPEG2000 was 15:1.
When processing one image at a time, there was a lot of unused processing capacity. For example, when compressing one ECW file at a time 700MB of total RAM was used; 25% of all eight cores were used; and 20% of the disk bandwidth was used.
By using this extra capacity with multiple simultaneous compressions we more effectively use our total available computing power. When compressing eight ECW files simultaneously 1.3GB of total RAM was used; 75% of all eight cores were used; and 90% of the disk bandwidth was used.
While the ERDAS IMAGINE batch engine does not limit its multi-core processing to the number of cores on the system, I stopped at eight because the disk capacity was filling up. I am convinced I could have pushed the number of simultaneous processes to 10 (or more) if I had added five more disks to fill the array container (but these additional five disks were not available).
ERDAS ER Mapper users will have slightly faster times when compressing one image, But, when running a batch of many images and more especially when running simultaneous processes, ERDAS IMAGINE will be significantly faster. In the future, we will incorporate into ERDAS IMAGINE what makes ERDAS ER Mapper faster when processing one image. Below are the times:
Looking at these numbers, we can see that a new day is rising for the remote sensing and GIS user community. Think for a moment, one ECW image was being completed every 0.9 seconds when running eight processes, and one JPEG2000 image was being completed every 1.2 seconds. This means you complete a large compression task in less time than it takes to each you to go eat lunch.
If you need a reason for your boss to spend money on an updated system, in the long term a fast processing configuration like the one I have tested is likely the least expensive productivity enhancement they can buy. All they need to do is determine how fast solutions are needed. Is compressing 2095 images over lunch fast enough?
Volume 1 Post Reference: http://field-guide.blogspot.com/2010/04/single-core-versus-multi-core-image.html
As you may recall, with the release of ERDAS IMAGINE 2010 this past December 2009, each IMAGINE Advantage license provides the customer the capability to batch process up to four simultaneous processes. If you have more IMAGINE Advantage licenses brokered using a floating license manager, each floating IMAGINE Advantage license can add four more simultaneous processes. In this experiment, I tested with one IMAGINE Advantage for 1 – 4 processes and another floating IMAGINE Advantage license for 5 – 8 processes.
With ERDAS IMAGINE 2010 v10.1 the latest version of the ECW JPEG2000 Codec SDK, v4.1 is used. This new version of the SDK has focused in JPEG2000 improvements. This new version of the ECW JPEG2000 Codec SDK will be made available to the market soon.
As discussed in vol 1, thanks to Dell, Inc. for providing the test system and SAM Inc for inspiring the software performance upgrades. The test system is configured as follows:
Dell Precision T7500
Processor: Dual Quad Core, 2.13GHz with 4MB Cache
System RAM: 4GB 1066MHz
Internal Controller: C4 SATA Non-RAID
Internal Disks: Four 1.5TB 7200 RPM SATA with 3.0Gbps, 16MB Data Burst Cache
External Storage: 15 Bay SAS/SATA Array
Array Controller: PERC 6/E SAS RAID, 256MB Memory
Array Disks: Ten 750GB 7200 RPM 3.0Gbps (note above, array can hold 15 disks)
Configuration: RAID 0
Operating System: Microsoft Windows Server 2008 x64, R2
I ran tests using the 2095 uncompressed strip GeoTIFFs of 5000 x 5000 x 3-bands x 8-bit data (each file is 73,282MB). The target compression ratio for both ECW and JPEG2000 was 15:1.
When processing one image at a time, there was a lot of unused processing capacity. For example, when compressing one ECW file at a time 700MB of total RAM was used; 25% of all eight cores were used; and 20% of the disk bandwidth was used.
By using this extra capacity with multiple simultaneous compressions we more effectively use our total available computing power. When compressing eight ECW files simultaneously 1.3GB of total RAM was used; 75% of all eight cores were used; and 90% of the disk bandwidth was used.
While the ERDAS IMAGINE batch engine does not limit its multi-core processing to the number of cores on the system, I stopped at eight because the disk capacity was filling up. I am convinced I could have pushed the number of simultaneous processes to 10 (or more) if I had added five more disks to fill the array container (but these additional five disks were not available).
ERDAS ER Mapper users will have slightly faster times when compressing one image, But, when running a batch of many images and more especially when running simultaneous processes, ERDAS IMAGINE will be significantly faster. In the future, we will incorporate into ERDAS IMAGINE what makes ERDAS ER Mapper faster when processing one image. Below are the times:
| # Processes | ECW Time | JPEG2000 Time |
1 | 2:51:15 | 5:46:44 |
2 | 1:29:27 | 2:44:44 |
3 | 0:43:48 | 1:58:24 |
4 | 0:41:19 | 1:47:03 |
5 | 0:41:37 | 1:06:55 |
6 | 0:35:49 | 0:55:03 |
7 | 0:33:01 | 0:45:51 |
8 | 0:31:53 | 0:43:14 |
Looking at these numbers, we can see that a new day is rising for the remote sensing and GIS user community. Think for a moment, one ECW image was being completed every 0.9 seconds when running eight processes, and one JPEG2000 image was being completed every 1.2 seconds. This means you complete a large compression task in less time than it takes to each you to go eat lunch.
If you need a reason for your boss to spend money on an updated system, in the long term a fast processing configuration like the one I have tested is likely the least expensive productivity enhancement they can buy. All they need to do is determine how fast solutions are needed. Is compressing 2095 images over lunch fast enough?
Volume 1 Post Reference: http://field-guide.blogspot.com/2010/04/single-core-versus-multi-core-image.html
Wednesday, April 21, 2010
Single core vs multi-core image processing (vol 1)
For many years the remote sensing and photogrammetry community have demanded more performance in processing of image data. (I still remember how excited I was when upgrading from an IBM XP with a 10MB hard disk to a Compaq 386 with a 300MB hard disk.) While, some functions continue to be limited by raw processing performance, most applications can receive massive benefit from a higher performance disk configuration. The next few post will discuss this topic.
Thanks to Dell, Inc. for the test system they provided. This is a cost-effective, yet solid performance server configuration. The test system is as follows:
Dell Precision T7500
Processor: Dual Quad Core, 2.13GHz with 4MB Cache
System RAM: 4GB 1066MHz
Internal Controller: C4 SATA Non-RAID
Internal Disks: Four 1.5TB 7200 RPM SATA with 3.0Gbps, 16MB Data Burst Cache
External Storage: 15 Bay SAS/SATA Array
Array Controller: PERC 6/E SAS RAID, 256MB Memory
Array Disks: Ten 750GB 7200 RPM 3.0Gbps
Configuration: RAID 0
Operating System: Microsoft Windows Server 2008 x64, R2
Since some people will read this blog months after these posts, I will not post prices. Notwithstanding my reluctance to mention prices, I will say that the performance boost will pay for itself. I suggest you look for the price this system on Dell’s website.
This effort actually began a few years ago over the Christmas Holidays when Chuck Patterson (SAM Inc. IT), Dell, Haiyan Qu (ERDAS Customer Support) and I got together to understand the performance of Dell’s EqualLogic Storage Array Network (SAN) device running ERDAS IMAGINE processes. That discussion was in the middle of ERDAS’ development of multi-core batch processing (former working name BatchX). With the completion of the multi-core batch processing effort at ERDAS, Dell and I restarted the testing with standard Disk Arrays. We will expand to SANs over the next few months.
As a benchmark to the speed of the system, I ran tests using the standard copy command in a .bat command file to determine the read-write speed of the configuration. Here are the speeds recorded on the system using 2095 uncompressed GeoTIFFs of 5000 x 5000 x 3-bands x 8-bit data (each file is 73,282MB):
0:14:26 Disk Array to Disk Array
0:23:01 Internal Disk to Disk Array
0:27:20 Disk Array to Internal Disk
0:40:01 Internal Disk, different disks
0:53:56 Internal Disk, same disks
As expected, the RAID 0 Disk Array really boosts performance. To be noted, RAID 0 is not redundant, and therefore long-term data storage on this configuration is not a good idea. But, processing data (especially writing) on a RAID 0 gives you powerful a performance boost.
In the next few posts, I will present file creation speeds of JPEG2000 and ECW in ERDAS IMAGINE 2010.1’s implementation of the ECW JPEG2000 Codec SDK v4.1 using this configuration. I hope you will find posts these useful.
Volume 2 post: http://field-guide.blogspot.com/2010/04/single-core-vs-multi-core-image.html
Thanks to Dell, Inc. for the test system they provided. This is a cost-effective, yet solid performance server configuration. The test system is as follows:
Dell Precision T7500
Processor: Dual Quad Core, 2.13GHz with 4MB Cache
System RAM: 4GB 1066MHz
Internal Controller: C4 SATA Non-RAID
Internal Disks: Four 1.5TB 7200 RPM SATA with 3.0Gbps, 16MB Data Burst Cache
External Storage: 15 Bay SAS/SATA Array
Array Controller: PERC 6/E SAS RAID, 256MB Memory
Array Disks: Ten 750GB 7200 RPM 3.0Gbps
Configuration: RAID 0
Operating System: Microsoft Windows Server 2008 x64, R2
Since some people will read this blog months after these posts, I will not post prices. Notwithstanding my reluctance to mention prices, I will say that the performance boost will pay for itself. I suggest you look for the price this system on Dell’s website.
This effort actually began a few years ago over the Christmas Holidays when Chuck Patterson (SAM Inc. IT), Dell, Haiyan Qu (ERDAS Customer Support) and I got together to understand the performance of Dell’s EqualLogic Storage Array Network (SAN) device running ERDAS IMAGINE processes. That discussion was in the middle of ERDAS’ development of multi-core batch processing (former working name BatchX). With the completion of the multi-core batch processing effort at ERDAS, Dell and I restarted the testing with standard Disk Arrays. We will expand to SANs over the next few months.
As a benchmark to the speed of the system, I ran tests using the standard copy command in a .bat command file to determine the read-write speed of the configuration. Here are the speeds recorded on the system using 2095 uncompressed GeoTIFFs of 5000 x 5000 x 3-bands x 8-bit data (each file is 73,282MB):
0:14:26 Disk Array to Disk Array
0:23:01 Internal Disk to Disk Array
0:27:20 Disk Array to Internal Disk
0:40:01 Internal Disk, different disks
0:53:56 Internal Disk, same disks
As expected, the RAID 0 Disk Array really boosts performance. To be noted, RAID 0 is not redundant, and therefore long-term data storage on this configuration is not a good idea. But, processing data (especially writing) on a RAID 0 gives you powerful a performance boost.
In the next few posts, I will present file creation speeds of JPEG2000 and ECW in ERDAS IMAGINE 2010.1’s implementation of the ECW JPEG2000 Codec SDK v4.1 using this configuration. I hope you will find posts these useful.
Volume 2 post: http://field-guide.blogspot.com/2010/04/single-core-vs-multi-core-image.html
Subscribe to:
Posts (Atom)