7 messages in net.java.dev.imageio-ext.devRE: OutOfMemoryError reading with MrS...| From | Sent On | Attachments |
|---|---|---|
| Nidel, Mike | Jul 17, 2008 10:22 am | |
| Daniele Romagnoli | Jul 18, 2008 6:52 am | |
| Nidel, Mike | Jul 18, 2008 7:03 am | |
| Simone Giannecchini | Jul 21, 2008 2:05 am | |
| Nidel, Mike | Jul 21, 2008 7:29 am | |
| Simone Giannecchini | Jul 23, 2008 5:24 pm | |
| Nidel, Mike | Jul 23, 2008 8:17 pm |
| Subject: | RE: OutOfMemoryError reading with MrSIDImageReader | |
|---|---|---|
| From: | Nidel, Mike (mike...@lmco.com) | |
| Date: | Jul 18, 2008 7:03:49 am | |
| List: | net.java.dev.imageio-ext.dev | |
I think a reply within 24 hours is pretty good, so no apology needed :)
I hadn't thought about the JAI ImageRead operation, but a solution that uses that would be very similar to implementing a RenderedImage, in the sense that it would require inserting another layer between my existing code and your code. That's OK, it's just not ideal. In the case of ImageRead, I would need a hard dependency on JAI ImageIO tools, which I very much want to avoid (for a few reasons). So given a choice between implementing my OWN RenderedImage wrapper for your ImageReader and implementing a solution that uses ImageRead (which ultimately returns a RenderedImage too), I would choose the first since there's no required dependency.
Essentially a possible GDALRenderedImage could do something like the following:
public Raster getTile(int tileX, int tileY) { Rectangle area = getTileRect(tileX, tileY); // do the math to figure out the tile region ImageReadParam param = new ImageReadParam(); param.setSourceRegion(area); // get only the area for the tile Raster result = myImageReader.readRaster(myImageIndex, param); return result; }
Does this sound right? The rest of the operations would have a default or standard implementation perhaps, just to pass through to the ImageReader.
Mike
________________________________
From: Daniele Romagnoli [mailto:dany...@gmail.com] Sent: Friday, July 18, 2008 9:53 AM To: de...@imageio-ext.dev.java.net Subject: Re: OutOfMemoryError reading with MrSIDImageReader
Mike, sorry for the late reply.
Did you considered the choice of using the JAI-ImageRead operation? It leverages on the Deferred Execution Model which allows to load data only when really needed. In such a case a getTile method will exactly get only the required part of data from the underlying gdal reader.
Anyway I will investigate on the implementation of the readAsRenderedImage method.
Regards, Daniele
On Thu, Jul 17, 2008 at 7:22 PM, Nidel, Mike <mike...@lmco.com> wrote:
I reported earlier that I had been having memory problems when reading MrSID. Here is an example portion of the stack trace:
java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:99) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288) at
it.geosolutions.imageio.gdalframework.GDALImageReader.readDatasetRaster( GDALImageReader.java:430) at
it.geosolutions.imageio.gdalframework.GDALImageReader.readDatasetRaster( GDALImageReader.java:325) at
it.geosolutions.imageio.gdalframework.GDALImageReader.read(GDALImageRead er.java:958) at
javax.imageio.ImageReader.readAsRenderedImage(ImageReader.java:1560)
I dug in a bit further and realized that our application assumes that a RenderedImage implementation is available to delay reading of actual pixels. But as the stack trace shows, readAsRenderedImage isn't implemented in GDALImageReader or MrSIDImageReader. So the default implementation in ImageReader gets called which just passes through to ImageReader.read() (which IS implemented in GDALImageReader).
The problem with this is that read() returns a BufferedImage which means that the entire extent of the image is read into memory. For large images this is a problem.
I suppose the GDAL ImageReaders are intended to be used by decoding a single BufferedImage for each subregion/tile/block of the image. This is certainly feasible but it doesn't match the expectation of my existing application, to which the ImageIO-ext library is just a plugin. Right now we use RenderedImage.getTile() for pixel access.
Do you have a recommended solution for this? Is there any alternative to just putting an extra layer in between my app and GDAL/ImageIO-ext to convert getTile(x,y) requests to read() calls?
thanks,
Mike
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-...@imageio-ext.dev.java.net For additional commands, e-mail: dev-...@imageio-ext.dev.java.net
-- ------------------------------------------------------- Eng. Daniele Romagnoli Software Engineer
GeoSolutions S.A.S. Via Carignoni 51 55041 Camaiore (LU) Italy
phone: +39 0584983027 fax: +39 0584983027 mob: +39 328 0559267
-------------------------------------------------------