Blocked images larger than a certain size will not render in Image Labeler

조회 수: 11 (최근 30일)
I am endeavouring to train a deep learning object detection network. For training data, I have large photomosaics that I would like to label using the Image Labeler app, and then ultimately create a Blocked Image Datastore with a Block Location Set argument to sample blocks from the large training images that are the appropriate input size for the network.
I appreciate that the Image Labeler app can import large images like these as blocked images, but I've found that above a certain size it seems to be unable to render them, and consequently I am unable to work with them in the Image Labeler.
Specifically, when things are working properly, if I import a blocked image into the lmage Labeler and proceed to zoom in from its initial zoom level, after several seconds at most the resolution of the image updates to the new zoom level. But above a certain image size, when I zoom in it just remains 'pixelated' from the previous zoom level's resolution, the resolution never updates to the new zoom level, and eventually after some time (several or more minutes) the MATLAB command window reports a graphics timeout error.
As it stands, my largest training image to not have this issue in Image Labeler is 15887 x 33197 pixels (≈ 527 million pixels), but as of my next size up image (18682 x 34701 pixels ≈ 648 million pixels), it will not render at higher zoom levels.
I'm not sure precisely which hardware resources are used to render images in Image Labeler, but my PC has 64 GB of RAM as well as an NVIDIA GeForce RTX 3060 with 12 GB of VRAM (and yes, I have the Parallel Computing Toolbox).
Any advice/insight?
  댓글 수: 5
Dominique Chabot
Dominique Chabot 2023년 5월 1일
Thank you for your additional replies.
Regarding how I've given the images to the Image Labeler, it doesn't seem to be able to import a blockedImageDatastore from the workspace, only a regular imageDatastore (and when I put the images into a regular imds and import them into the Labeler, it doesn't even suggest converting them to blocked images; just goes ahead and imports them as regular images, so that's a non-starter), so I've been importing them from file (in which case it does suggest converting them to blocked images, though you don't get to specify a block size; my understanding has been that it automatically sets the block size based on the size of the image and maybe the zoom level).
As for the image properties, here are those of the smallest image I have that gives me rendering issues in the Image Labeler, after converting it to a blocked image in the workspace:
blockedImage with properties:
Read only properties
Source: "G:\Imagery for processing\2022 CWS seabird cliff surveys\MATLAB\Training images\DSC02051 Panorama.jpg"
Adapter: [1×1 images.blocked.GenericImage]
Size: [34701 18682 3]
SizeInBlocks: [1 1 1]
ClassUnderlying: "uint8"
In any case, I thought the whole point of blocked images--as described by the Image Labeler app itself--is to "load only the data being looked at" so as not to strain the memory in the PC or graphics card...

댓글을 달려면 로그인하십시오.

채택된 답변

Ashish Uthama
Ashish Uthama 2023년 5월 3일
Thanks for sharing the blockedImage representation - This is a JPG file. Which means there is really no way to do blocked IO (i.e read only parts of the file). Since its a compressed format, the whole file needs to be uncompressed to access any small region of the image.
You could convert this file (one time hit) using something like: (choose a blocksize the same as the image size you want to train your network on, that will help IO performance down the line)
makeMultiLevel2D(bim, BlockSize=[1024 1024], Scales=[1 .1], OutputLocation="withOverview.tif", Adapter=images.blocked.TIFF)
Loading this TIFF file into the app should hopefully give you better performance. It will also help when you get to the blockedImageDatastore workflow.
  댓글 수: 2
Ashish Uthama
Ashish Uthama 2023년 5월 8일
Many things to unpack :)
TIFF vs JPEG - JPEG is 'lossy' compression, details are lost. Which is why it can compress much better than LZW in TIFF (lossless). Depending on your experiment, the loss in quality may not matter. There is an option to have each tile in TIFF comrpessed with JPEG which gives you the best of both worlds. Not sure if your image stitching software has than option to expore. You can create that with blockedImage/write though - this should reduce file size significantly.
adapter = images.blocked.TIFF;
adapter.Compression = Tiff.Compression.JPEG;
write(bim,'out.tif', BlockSize = [1024, 1024],Adapter=adapter)
About long time to import - if your generating software has a capability to create pyramid versions (which is what the makeMultiLevel2D function does) that would help significantly. TIFFs support this format, JPG do not. Having a low resolution version that can be quickly loaded helps the labeller app quickly show you the overiew and only load the finer details when you get down to the block level.
Good point about the suggestion from Image Labeler. That catch is that the boundary of when a JPEG file is 'too large' is highly dependent on a user's machine, ability to change format and a user's patience :) We can definitely add more color to that suggestion.
Not familiar, but using a single labelling project should be the right thing to do.

댓글을 달려면 로그인하십시오.

추가 답변 (3개)

Matt J
Matt J 2023년 5월 1일
I could of course break the images up into smaller images but that would make my workflow less efficient
Breaking them up for the purposes of network training shouldn't impact your workflow. After the network is trained, you can use the original image size for inference.
  댓글 수: 1
Dominique Chabot
Dominique Chabot 2023년 5월 1일
Yes, I suppose that would work, though I'm still curious whether this is a known limitation for blocked images in the Image Labeler app (no image size limitation appears to be mentioned in the app's documentation) or whether it may have something to do with my PC's hardware being insufficient.

댓글을 달려면 로그인하십시오.

Matt J
Matt J 2023년 5월 1일
편집: Matt J 님. 2023년 5월 1일
my understanding has been that it automatically sets the block size based on the size of the image and maybe the zoom level
Perhaps it should, but my guess would be that it chooses it no differently from how the blockedImage constructor chooses it and, as you can see from the object properties, the default SizeInBlocks that it has chosen is [1,1,1]. In other words, the software is assuming that a [34701 18682 3] chunk of data is just fine for the memory resources of your computer. No need to split it up.
Are you able to import a blockedImage object to the app? If so, you should probably import to the workspace as a blockedImage object with the blockSize chosen by you directly, e.g.,
bim = blockedImage(Source, blockSize=[1000,1000]);
  댓글 수: 3
Matt J
Matt J 2023년 5월 1일
I think we may have exhausted all avenues, but you could always try Tech Support, to see if there's something we've overlooked. If nothing else, they should probably be informed about these limitations.

댓글을 달려면 로그인하십시오.

Matt J
Matt J 2023년 5월 1일
편집: Matt J 님. 2023년 5월 1일
One other possible workaround is to convert the image to grayscale, e.g., using the ReadFcn property of the imageDataStore, for the purposes of labeling only. I don't know if the loss of color affects your ability to recognize targets for labeling purposes. If not, it will cut down memory requirements by a factor of 3.
I would still recommend you chop up the images in any case. I would expect it to make training go much faster.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by