pixelLabelImageDatastore not partitionable despite underlying imageDatastore and pixelLabelDatastore are

조회 수: 2 (최근 30일)
I have a pixelLabelImageDatastore constructed with: pximds = pixelLabelImageDatastore(imds, pxds). Both imds and pxds are partitionable but the resulting pximds is not, which stops me from using multi-gpu.
To me it looks very strange. How do I make a partitionable pixelLabelImageDatastore?
>> isPartitionable(imds)
ans =
>> isPartitionable(pxds)
ans =
>> pximds = pixelLabelImageDatastore(imds, pxds);
>> isPartitionable(pximds)
ans =

채택된 답변

Joss Knight
Joss Knight 2022년 2월 24일
For parallel training of a semantic segmentation network, you're supposed to use combine to join the two input datastores rather than using a pixelLabelImageDatastore.

추가 답변 (1개)

yanqi liu
yanqi liu 2022년 2월 21일
yes,sir,may be use partitionCamVidData,such as
function [imdsTrain, imdsVal, imdsTest, pxdsTrain, pxdsVal, pxdsTest] = partitionCamVidData(imds,pxds)
% Partition CamVid data by randomly selecting 60% of the data for training. The
% rest is used for testing.
% Set initial random state for example reproducibility.
numFiles = numel(imds.Files);
shuffledIndices = randperm(numFiles);
% Use 60% of the images for training.
numTrain = round(0.60 * numFiles);
trainingIdx = shuffledIndices(1:numTrain);
% Use 20% of the images for validation
numVal = round(0.20 * numFiles);
valIdx = shuffledIndices(numTrain+1:numTrain+numVal);
% Use the rest for testing.
testIdx = shuffledIndices(numTrain+numVal+1:end);
% Create image datastores for training and test.
trainingImages = imds.Files(trainingIdx);
valImages = imds.Files(valIdx);
testImages = imds.Files(testIdx);
imdsTrain = imageDatastore(trainingImages);
imdsVal = imageDatastore(valImages);
imdsTest = imageDatastore(testImages);
% Extract class and label IDs info.
classes = pxds.ClassNames;
labelIDs = camvidPixelLabelIDs();
% Create pixel label datastores for training and test.
trainingLabels = pxds.Files(trainingIdx);
valLabels = pxds.Files(valIdx);
testLabels = pxds.Files(testIdx);
pxdsTrain = pixelLabelDatastore(trainingLabels, classes, labelIDs);
pxdsVal = pixelLabelDatastore(valLabels, classes, labelIDs);
pxdsTest = pixelLabelDatastore(testLabels, classes, labelIDs);
  댓글 수: 1
Jari Manni
Jari Manni 2022년 2월 21일
Hi, thanks for the answer. But I think this is a separate issue. My problem is not on the train set, val set, test set partitioning, but related to multi gpu training where training set is splited into different GPU cards.
When I specify 'ExecutionEnvironment', 'multi-gpu', I get the following error:
'Error using trainNetwork (line 184)
The input datastore is not Partitionable and does not support parallel operations.
UnderlyingCause: [1×1 MException]
identifier: 'nnet_cnn:cnn:GeneralDatastoreDispatcher:NonPartitionableDatastore'
message: 'The input datastore is not Partitionable and does not support parallel operations.'
cause: {}
stack: [2×1 struct]
Correction: []

댓글을 달려면 로그인하십시오.


Help CenterFile Exchange에서 Parallel and Cloud에 대해 자세히 알아보기




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by