More on Consolidated Mosaic

Well, it has become well known by now that MosaicPro will be included as a native part of IMAGINE Advantage and LPS Core in ERDAS IMAGINE 2010, due Fall 2009. The new MosaicPro had an internal working name of Consolidated Mosaic. MosaicTool, MosaicPro, lessons from ER Mapper and more were used to create the new MosaicPro (http://field-guide.blogspot.com/2009/06/consolidated-mosaic.html). Moreover, we have optimized MosaicPro to be much more RAM and CPU scalable, and that it touches each pixel the fewest times possible.

Yet, let me yell this from the house top.... all ERDAS IMAGINE and LPS customers with current Software Maintenance (SWM) will be receiving this 'new' MosaicPro in version 2010. It is a real gem. You will be impressed.

Here are some early tests:
Dual-core 2 GHz CPU, 4GB RAM, standard disks, Vista Business

Load 3840 images:
9.3.2 16:40 minutes; 463 MB RAM
2010 3:05 minutes; 98 MB RAM

Load 4896 images:
9.3.2 35:06 minutes; 669 MB RAM
2010 4:16 minutes; 113 MB RAM

Cutline Generation on 4896 images:
9.3.2 45:47 minutes; 1,606 MB RAM
2010 01:41 minutes; 125 MB RAM

Mosaicking 127 images:
9.3.2 41:41 minutes
2010 29:11 minutes

Dual Quad-core 2.328 GHz CPU, 16GB RAM, average speed disk array, XP-64
Mosaicking 3982 images:
9.3.2 Not possible
2010 >700GB output file in under 2.5 days

Mosaicking 1147 images:
9.3.2 Not possible
2010 >2.6TB output file in under 5.5 days

All processes used a maximum of just over 800MB of RAM. We researched using more RAM with MosaicPro and found it does not help. This is because we have tuned it so thoroughly that the bottleneck is now the operating system’s thread handling, and hard disk I/O. So, it is time to test fast disks.

But, when using the new added ECW / JPEG2000 direct-write from MosaicPro, MosaicPro will need more RAM. We are working on that as well. We expect we can protect the speed while shrinking down the memory requirement significantly. We expect this will occur in version 2010.1 (Sprilg 2010) or version 2011 (Fall 2010).

Now, we are ready to take MosaicPro to 64-bit. As you can see, when ERDAS goes to 64-bit, we will really take advantage of what 64-bit can really do. Many companies have used a 64-bit port to mask their sloppy coding by accessing more RAM. Not at ERDAS! We know there are a lot of issues surrounding true speed improvement and large file handling. We want our loyal customers to have it all! (http://field-guide.blogspot.com/2009/02/benefits-of-64-bit-architecture-in.html)

Personally, I enjoy looking at ERDAS IMAGINE as a CPU race-car entry. Maybe we can get Intel to test MosaicPro as it does ECW in its performance testing matrix?

Consolidated Mosaic

Consolidated Mosaic is a working name for a new mosaic paradigm that will be introduced in ERDAS IMAGINE 2010. Consider the different tools ERDAS, Inc. has for mosaicking images:

• ERDAS IMAGINE's MosaicTool (original mosaic tools, good defaults options)
• ERDAS MosaicPro (advanced MosaicTool, easy cutlines, ortho-correct from block files )
• ERDAS ER Mapper Mosaic (fast virtual mosaic, a lot of capacity, limited capability)
• ERDAS ER Mapper Color Balance (fast color balance, limited to true color)
• ERDAS Image Compressor (fast ECW and JPEG 2000 compression)
• ERDAS IMAGINE's MosaicDirect (Wizard to feed to MosaicPro and batch)
• ERDAS IMAGINE's MosaicWizard (Wizard to process mosaic)
• ERDAS IMAGINE's Virtual Mosaic (more capability than ER Mapper Mosaic, less capacity)

What if we combined these products in a single product? What if you could mosaic >2.5 terapixels of data straight into a single >2.5 terapixel IMG, or to a 20:1 compressed ECW, or to a lossless compressed JPEG2000 image. What if you could break that >2.5 terapixel mosaic into tiles with your shapefiles (and it has no temp files)? What if you could do all this within a 32-bit operating system environment?

If you think this can help you, keep your eyes open for a WebEx or an erdas labs discussion on this topic very soon.

I gave you a hint of where we were going when I asked Hammad to post to The Field Guide in: http://field-guide.blogspot.com/2009/02/benefits-of-64-bit-architecture-in.html

Create an IMAGINE Image File List (.fls) for use in MosaicTool and MosaicPro.

I originally created this procedure while at the Georgia Institute of Technology (Georgia Tech) Center for Geographic Information Systems.

The structure of .fls format is documented in ERDAS IMAGINE as follows:
n number-of-images
i reference-image
0 imagename_1
1 imagename_2
.
.
.
n-1 imagename_n

An example is as follows:
3 number-of-images
1 reference-image
0 R:\Data\wasia1_mss.img
1 R:\Data\wasia2_mss.img
2 R:\Data\wasia3_tm.img

How do I create a .fls for all TIFF images in a directory and in all sub-directories?

First, use a DOS Command Window
1. Open a DOS Command Window
2. Change to the appropriate directory
3. Use "dir" command as follows: dir *.tif /b /s > mosaic-list.txt
(The /s parameter looks in sub-directories; the /b makes the list 'bare')

Next, use MS Excel to organize the list
4. Open "mosaic-list.txt" in Excel
5. Insert a column to the left of the list
6. Insert 2 rows at the top of the list
7. Add 0 in the first column of the third row
8. Add =A3+1 to the first column of the forth row
9. Copy formula from #8 above to all rows with images
10. Copy the number for the last image to the first column of the first row
11. Select an image number for the reference image (can change in Mosaic later) and put in the first column of the second row.
12. Find / Replace all \" with "/", in the image column.
13. Save as the file to a text file to your desired txt filename.

Next, use MS Word to remove tabs introduced by Excel:
14. Replace tab "^t" with a space " " (do not enter quote marks)
15. Save as text with the extension of .fls

Finally, open in Mosaic Tool or Mosaic.