Main Content

estimateNetworkOutputBounds

Estimate output bounds of deep learning network

Since R2022b

collapse all in page

    Syntax

    [YLower,YUpper] = estimateNetworkOutputBounds(net,XLower,XUpper)
    result = estimateNetworkOutputBounds(___,Name=Value)

    Description

    [YLower,YUpper] = estimateNetworkOutputBounds(net,XLower,XUpper) estimates the lower and upper output bounds, YLower and YUpper, respectively, of the network net for input within the bounds specified by XLower and XUpper.

    The function estimates the range of output values that the network returns when the input is between the specified lower and upper bounds. Use this function to estimate how sensitive the network predictions are to input perturbation.

    The estimateNetworkOutputBounds function requires the Deep Learning Toolbox Verification Library support package. If this support package is not installed, use the Add-On Explorer. To open the Add-On Explorer, go to the MATLAB® Toolstrip and click Add-Ons > Get Add-Ons.

    example

    result = estimateNetworkOutputBounds(___,Name=Value) verifies the adversarial robustness with additional options specified by one or more name-value arguments.

    Examples

    collapse all

    This example uses:

    Open Live Script

    Estimate the output bounds for an image regression network.

    Load a pretrained regression network. This network is a dlnetwork object that has been trained to predict the rotation angle of images of handwritten digits.

    load("digitsRegressionNetwork.mat");

    Load the test data.

    [XTest,~,TTest] = digitTest4DArrayData;

    Select the first ten images.

    X = XTest(:,:,:,1:10);
T = TTest(1:10);

    Convert the test images to dlarray objects.

    X = dlarray(X,"SSCB");

    Estimate the output bounds for an input perturbation between –0.01 and 0.01 for each pixel. Create lower and upper bounds for the input.

    perturbation = 0.01;
XLower = X - perturbation;
XUpper = X + perturbation;

    Estimate the output bounds for each input.

    [YLower,YUpper] = estimateNetworkOutputBounds(net,XLower,XUpper);

    The output bounds are dlarray objects. To plot the output bounds, first extract the data using extractdata.

    YLower = extractdata(YLower);
YUpper = extractdata(YUpper);

    Visualize the output bounds.

    figure
hold on
for i = 1:10
    plot(i,T(i),"ko")
    line([i i],[YLower(i) YUpper(i)],Color="b")
end
hold off
xlim([0 10])
xlabel("Observation")
ylabel("Angle of Rotation")
legend(["True value","Output bounds"])

    Figure contains an axes object. The axes object with xlabel Observation, ylabel Angle of Rotation contains 20 objects of type line. One or more of the lines displays its values using only markers These objects represent True value, Output bounds.

    Input Arguments

    collapse all

    Network, specified as an initialized dlnetwork object. To initialize a dlnetwork object, use the initialize function.

    The function supports networks with these layers:

    The function does not support networks with multiple inputs and multiple outputs.

    The function estimates the output bounds using the final layer in the network. For most applications, use the final fully connected layer when computing the output bounds. If your network has a different layer as its final layer, remove the layer before calling the function.

    Input lower bound, specified as a formatted dlarray object. For more information about dlarray formats, see the fmt input argument of dlarray.

    The lower and upper bounds, XLower and XUpper, must have the same size and format. The function computes the results across the batch ("B") dimension of the input lower and upper bounds.

    Input upper bound, specified as a formatted dlarray object. For more information about dlarray formats, see the fmt input argument of dlarray.

    The lower and upper bounds, XLower and XUpper, must have the same size and format. The function computes the results across the batch ("B") dimension of the input lower and upper bounds.

    Name-Value Arguments

    Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

    Example: estimateNetworkOutputBounds(net,XLower,XUpper,label,MiniBatchSize=32,ExecutionEnvironment="multi-gpu") estimates the network output bounds using a mini-batch size of 32 and using multiple GPUs.

    Since R2023b

    Size of the mini-batch to use when estimating network output bounds, specified as a positive integer.

    Larger mini-batch sizes require more memory, but can lead to faster computations.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Since R2024a

    Hardware resource for estimating network output bounds, specified as one of these values:

    • "auto" – Use a local GPU if one is available. Otherwise, use the local CPU.

    • "cpu" – Use the local CPU.

    • "gpu" – Use the local GPU.

    • "multi-gpu" – Use multiple GPUs on one machine, using a local parallel pool based on your default cluster profile. If there is no current parallel pool, the software starts a parallel pool with pool size equal to the number of available GPUs.

    • "parallel-auto" – Use a local or remote parallel pool. If there is no current parallel pool, the software starts one using the default cluster profile. If the pool has access to GPUs, then only workers with a unique GPU perform the computations and excess workers become idle. If the pool does not have GPUs, then the computations take place on all available CPU workers instead.

    • "parallel-cpu" – Use CPU resources in a local or remote parallel pool, ignoring any GPUs. If there is no current parallel pool, the software starts one using the default cluster profile.

    • "parallel-gpu" – Use GPUs in a local or remote parallel pool. Excess workers become idle. If there is no current parallel pool, the software starts one using the default cluster profile.

    The "gpu", "multi-gpu", "parallel-auto", "parallel-cpu", and "parallel-gpu" options require Parallel Computing Toolbox™. To use a GPU for deep learning, you must also have a supported GPU device. For information on supported devices, see GPU Computing Requirements (Parallel Computing Toolbox). If you choose one of these options and Parallel Computing Toolbox or a suitable GPU is not available, then the software returns an error.

    For more information on when to use the different execution environments, see Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud.

    Output Arguments

    collapse all

    Output lower bound, returned as a formatted dlarray object. For more information about dlarray formats, see the fmt input argument of dlarray.

    The function estimates the output bounds for each observation across the batch ("B") dimension. If you supply k upper bounds and lower bounds, then YLower contains k output lower bounds. For more information, see Algorithms.

    Output upper bound, returned as a formatted dlarray object. For more information about dlarray formats, see the fmt input argument of dlarray.

    The function estimates the output bounds for each observation across the batch ("B") dimension. If you supply k upper bounds and lower bounds, then YUpper contains k output upper bounds. For more information, see Algorithms.

    Algorithms

    Let X be the input for which you want to estimate the output bounds. To use the estimateNetworkOutputBounds function, you must specify a lower and upper bound for the input. For example, let ϵ be a small perturbation. You can define a lower and upper bound for the input as Xlower=X−ϵ and Xupper=X+ϵ, respectively.

    To estimate the output bounds for the network net and the input bounds Xlower and Xupper, the function performs these steps.

    1. Create an input set using the lower and upper input bounds.

    2. Pass the input set through the network and return an output set. To reduce computational overhead, the function performs abstract interpretation by approximating the output of each layer using the DeepPoly [2] method.

    3. Return the minimum and maximum estimated output values for the input set.

    If you specify multiple pairs of input lower and upper bounds, then the function estimates the output bounds for each pair of input bounds.

    Note

    Soundness with respect to floating point: In rare cases, floating-point rounding errors can accumulate which can cause the network output to be outside the computed bounds and the verification results to be different. This can also be true when working with networks you produced using C/C++ code generation.

    References

    [1] Goodfellow, Ian J., Jonathon Shlens, and Christian Szegedy. “Explaining and Harnessing Adversarial Examples.” Preprint, submitted March 20, 2015. https://arxiv.org/abs/1412.6572.

    [2] Singh, Gagandeep, Timon Gehr, Markus Püschel, and Martin Vechev. “An Abstract Domain for Certifying Neural Networks”. Proceedings of the ACM on Programming Languages 3, no. POPL (January 2, 2019): 1–30. https://doi.org/10.1145/3290354.

    Extended Capabilities

    Version History

    Introduced in R2022b

    expand all

    See Also

    | |

    Topics