Why would the file size of a deep learning gradient become much bigger after saving as a .mat file?
조회 수: 8(최근 30일)
표시 이전 댓글
SC 2019년 12월 2일
답변: Joss Knight 2019년 12월 3일
I have a variable gradients which is the gradient of a deep learning model. From the code "whos gradients" you can see that it only requires 5742 bytes (i.e. 5.7 kB) to store. However, when I tried to save it as a .mat file, the file size becomes 13320098 bytes (i.e. 13.3 mB. Please refer to the code "file_size=file.bytes" ), which is more than 2000 times larger. May I ask the reason why, and how can I save the file with a size similar to 5.7kB?
%% Define Network Architecture
layers = [
imageInputLayer([1 1 100],'Normalization','none','Name','in')
MyLGraph = layerGraph(layers);
myDLnet = dlnetwork(MyLGraph);
gradients = dlfeval(@modelGradients, myDLnet, dlZ, Y);
function [gradients] = modelGradients(myModel, modelInput, CorrectLabels)
[modelOutput,state] = forward(myModel,modelInput);
loss = -sum(sum(CorrectLabels_transpose.*log(sigmoid(modelOutput_mean/100))));
gradients = dlgradient(loss, myModel.Learnables);
function [dlZ, Y]=get_dlZ_Y()
rng(123); % seed
Z = randn(1,1,100,100,'single');
Y = randn(1,100,'single');
% Convert mini-batch of data to dlarray specify the dimension labels
% 'SSCB' (spatial, spatial, channel, batch).
dlZ = dlarray(Z, 'SSCB');
% If training on a GPU, then convert data to gpuArray.
if (executionEnvironment == "auto" && canUseGPU) || executionEnvironment == "gpu"
dlZ = gpuArray(dlZ);
댓글 수: 1
Walter Roberson 2019년 12월 2일
I wonder if you are configured to default to -v7.3 files for save()? There is a preference for that.
Joss Knight 2019년 12월 3일
The difference is that whos is unable to account for the fact that the data is all stored on the GPU, and is only showing CPU memory. Add the following
gradients = dlupdate(@gather, gradients);
and you will see parity between the numbers.
The answer is - you can't save 3.6 million 32-bit numbers in 5.7 kilobytes, no matter what magic you employ!
댓글 수: 0
Find more on Image Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!