How can I use deepNetworkQuantizer with my own data and network.

Hi,

Currently I am trying to learn about how to use the deepNetworkQuantizer. Having a pre-trained dlnetwork, when I try to start validating according to the tutorial, an error occured:

Here are some information about my project.

• my network is a simple classification network, which takes a 1-D vector as input and outputs the result from a Softmax Layer
• my dataset can be considered as a 2-D matrix, and the ground truth would also be a 2-D matrix which aligned with one-hot vectors.
• In the deepNetworkQuantizer, my calibration data is a 2-D matrix, in which each row corresponds to an input feature vector
• In the deepNetworkQuantizer, my validation data is also a 2-D matrix, but another part of the dataset.

When i click the validate, the error occurred.

How can i solve this? And i also have some questions.

1. Is there a specific datatype for the Quantizer input? As i saw in the docs that it mostly based on `augmentedImageDatastore.
2. How can i write my custom metric function. I read a sample on https://ww2.mathworks.cn/help/deeplearning/ref/deepnetworkquantizer-app.html#mw_12115181-bf7e-4148-88ee-8d7688ca16f6 , but i was quite confused about the input of the function.

function accuracy = hComputeModelAccuracy(predictionScores, net, dataStore)
%% Computes model-level accuracy statistics
    
    % Load ground truth.
    tmp = readall(dataStore);
    groundTruth = tmp.response;
    
    % Compare predicted label with actual ground truth. 
    predictionError = {};
    for idx=1:numel(groundTruth)
        [~, idy] = max(predictionScores(idx,:)); 
        yActual = net.Layers(end).Classes(idy);
        predictionError{end+1} = (yActual == groundTruth(idx)); %#ok
    end
    
    % Sum all prediction errors.
    predictionError = [predictionError{:}];
    accuracy = sum(predictionError)/numel(predictionError);
end

Can i consider it as a template about writing a custom metric fucntion? And how can the function get dataStore which contains the ground truth label.

3. How can i use the default metric function (top-1 accuracy).How should my data be?

Thanks in advanced for your help!

Answer 1

[TED MOSBY]

Hi @Haoming,

The error message "numeric data not supported for validate" typically occurs when the validation data passed to the deepNetworkQuantizer is in a format that it cannot process. This can happen if the data is provided as a standard numeric matrix or array.

You must use a ‘datastore’ object for that, I found this in the documentation:

In this case, since your data is a 2-D numeric array, you can use an ‘arrayDatastore’ to handle the validation data.

% Assuming your data is a matrix where each row is an input vector
calibrationData = arrayDatastore(yourCalibrationMatrix, 'IterationDimension', 1);
validationData = arrayDatastore(yourValidationMatrix, 'IterationDimension', 1);

Please go through the complete page link I have attached below and you will have a clear understanding of what kind of input is sent in which form according to your dataset:

www.mathworks.com/help/deeplearning/ug/prepare-data-for-quantizing-networks.html

This link will resolve your error query and your first question regarding quantizer inputs.

To answer your query regarding the inputs of the custom function, the meanings are as below:

• predictionScores: The predicted scores from the network.
• net: The network object.
• dataStore: The datastore containing the ground truth labels.

Read all data from ‘dataStore’ to get the ground truth labels:

tmp = readall(dataStore);
groundTruth = tmp.response;  % you can adjust 'response' to the actual name of your labels variable

For your query regarding the Top-1 accuracy, the app uses this metric by default and your data should be set up such that both your input data and label are in a compatible datastore, such as ‘arrayDatastore’.

Refer to this link: www.mathworks.com/help/deeplearning/ref/deep.metric.accuracymetric.html#mw_5ab5f0dc-ff92-481d-969d-7f56d1550a78

Hope this helps!