This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English version of the page.

Note: This page has been translated by MathWorks. Click here to see
To view all translated materials including this page, select Country from the country navigator on the bottom of this page.

Code Generation for Denoising Deep Neural Network

This example shows how to generate CUDA® MEX from MATLAB® code and denoise grayscale images by using the denoising convolutional neural network (DnCNN [1]). Denoising network can be used to estimate noise in a noisy image and then remove it to obtain a denoised image.

Prerequisites

  • CUDA enabled NVIDIA® GPU with compute capability 3.2 or higher.

  • NVIDIA CUDA toolkit and driver.

  • NVIDIA cuDNN library v7 or higher.

  • Environment variables for the compilers and libraries. For information on the supported versions of the compilers and libraries, see Third-party Products. For setting up the environment variables, see Environment Variables.

  • GPU Coder™ for generating CUDA code.

  • Deep Learning Toolbox™ for using SeriesNetwork or DAGNetwork objects.

  • Image Processing Toolbox™ for reading and displaying images.

  • GPU Coder Interface for Deep Learning Libraries support package. To install this support package, use the Add-On Explorer.

Verify the GPU Environment

Use the coder.checkGpuInstall function and verify that the compilers and libraries needed for running this example are set up correctly.

coder.checkGpuInstall('gpu','codegen','cudnn','quiet');

Load Noisy Image

Load a noisy grayscale image onto the workspace and display the image.

noisyI = imread('noisy_cameraman.png');
figure
imshow(noisyI);
title('Noisy Image');

Get the Pretrained Denoising Network

Call the getDenoisingNetwork function to get a pretrained image denoising deep neural network.

net = getDenoisingNetwork;

The getDenoisingNetwork function returns a pretrained DnCNN [1] that can be used to detect additive white Gaussian noise (AWGN) with unknown levels. The network is a feed-forward denoising convolutional network that implements a residual learning technique to predict a residual image. In other words, DnCNN [1] computes the difference between a noisy image and the latent clean image.

The network contains 59 layers including convolution, batch normalization, and regression output layers.

net.Layers
ans = 

  59x1 Layer array with layers:

     1   'InputLayer'             Image Input           50x50x1 images
     2   'Conv1'                  Convolution           64 3x3x1 convolutions with stride [1  1] and padding [1  1  1  1]
     3   'ReLU1'                  ReLU                  ReLU
     4   'Conv2'                  Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
     5   'BNorm2'                 Batch Normalization   Batch normalization with 64 channels
     6   'ReLU2'                  ReLU                  ReLU
     7   'Conv3'                  Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
     8   'BNorm3'                 Batch Normalization   Batch normalization with 64 channels
     9   'ReLU3'                  ReLU                  ReLU
    10   'Conv4'                  Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    11   'BNorm4'                 Batch Normalization   Batch normalization with 64 channels
    12   'ReLU4'                  ReLU                  ReLU
    13   'Conv5'                  Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    14   'BNorm5'                 Batch Normalization   Batch normalization with 64 channels
    15   'ReLU5'                  ReLU                  ReLU
    16   'Conv6'                  Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    17   'BNorm6'                 Batch Normalization   Batch normalization with 64 channels
    18   'ReLU6'                  ReLU                  ReLU
    19   'Conv7'                  Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    20   'BNorm7'                 Batch Normalization   Batch normalization with 64 channels
    21   'ReLU7'                  ReLU                  ReLU
    22   'Conv8'                  Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    23   'BNorm8'                 Batch Normalization   Batch normalization with 64 channels
    24   'ReLU8'                  ReLU                  ReLU
    25   'Conv9'                  Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    26   'BNorm9'                 Batch Normalization   Batch normalization with 64 channels
    27   'ReLU9'                  ReLU                  ReLU
    28   'Conv10'                 Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    29   'BNorm10'                Batch Normalization   Batch normalization with 64 channels
    30   'ReLU10'                 ReLU                  ReLU
    31   'Conv11'                 Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    32   'BNorm11'                Batch Normalization   Batch normalization with 64 channels
    33   'ReLU11'                 ReLU                  ReLU
    34   'Conv12'                 Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    35   'BNorm12'                Batch Normalization   Batch normalization with 64 channels
    36   'ReLU12'                 ReLU                  ReLU
    37   'Conv13'                 Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    38   'BNorm13'                Batch Normalization   Batch normalization with 64 channels
    39   'ReLU13'                 ReLU                  ReLU
    40   'Conv14'                 Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    41   'BNorm14'                Batch Normalization   Batch normalization with 64 channels
    42   'ReLU14'                 ReLU                  ReLU
    43   'Conv15'                 Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    44   'BNorm15'                Batch Normalization   Batch normalization with 64 channels
    45   'ReLU15'                 ReLU                  ReLU
    46   'Conv16'                 Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    47   'BNorm16'                Batch Normalization   Batch normalization with 64 channels
    48   'ReLU16'                 ReLU                  ReLU
    49   'Conv17'                 Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    50   'BNorm17'                Batch Normalization   Batch normalization with 64 channels
    51   'ReLU17'                 ReLU                  ReLU
    52   'Conv18'                 Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    53   'BNorm18'                Batch Normalization   Batch normalization with 64 channels
    54   'ReLU18'                 ReLU                  ReLU
    55   'Conv19'                 Convolution           64 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    56   'BNorm19'                Batch Normalization   Batch normalization with 64 channels
    57   'ReLU19'                 ReLU                  ReLU
    58   'Conv20'                 Convolution           1 3x3x64 convolutions with stride [1  1] and padding [1  1  1  1]
    59   'FinalRegressionLayer'   Regression Output     mean-squared-error with response 'Response'

About the 'denoisenet_predict' Function

The denoisenet_predict function takes a noisy image input and returns a denoised image using a pretrained denoising network.

The function loads the network object returned by getDenoisingNetwork into a persistent variable mynet. On subsequent calls to the function, the persistent object is reused for prediction.

type denoisenet_predict
% Copyright 2018 The MathWorks, Inc.

function I = denoisenet_predict(in)
%#codegen

% A persistent object mynet is used to load the SeriesNetwork network object.
% At the first call to this function, the persistent object is constructed and
% setup. When the function is called subsequent times, the same object is reused 
% to call predict on inputs, thus avoiding reconstructing and reloading the
% network object.

persistent mynet;


if isempty(mynet)   
    mynet = coder.loadDeepLearningNetwork('getDenoisingNetwork', 'DnCNN');
end

% The activations methods extracts the output from the last layer. The
% 'OutputAs' 'channels' name-value pair argument is used inorder to call
% activations on an image whose input dimensions are greater than or equal
% to the network's imageInputLayer.InputSize.

res = mynet.activations(in, 59,'OutputAs','channels');


% Once the noise is estimated, we subtract the noise from the original
% image to obtain a denoised image.

I = in - res;
  

Here, the activations method is invoked with the layer numeric index as 59 to extract the activations from the final layer of the network. The 'OutputAs' 'channels' name-value pair argument is used to compute activations on images larger than the imageInputLayer.InputSize of the network.

The activations method returns an estimate of the noise in the input image using the pretrained denoising image.

Once the noise is estimated, we subtract the noise from the original image to obtain a denoised image.

Run MEX Code Generation for 'denoisenet_predict' Function

To generate CUDA code from design file denoisenet_predict.m, reate a GPU code configuration object for a MEX target and set the target language to C++. Use the coder.DeepLearningConfig function to create a CuDNN deep learning configuration object and assign it to the DeepLearningConfig property of the GPU code configuration object. Run the codegen command specifying an input of size [256,256]. This value corresponds to the size of the noisy image that we intend to denoise.

cfg = coder.gpuConfig('mex');
cfg.TargetLang = 'C++';
cfg.DeepLearningConfig = coder.DeepLearningConfig('cudnn');
codegen -config cfg denoisenet_predict -args {ones(256,256,'single')} -report
Code generation successful: To view the report, open('codegen/mex/denoisenet_predict/html/report.mldatx').

Run the Generated MEX

The DnCNN [1] is trained on input images having an input range [0,1]. Call im2single on noisyI to rescale the values from [0,255] to [0,1].

Call denoisenet_predict on the rescaled input image.

denoisedI = denoisenet_predict_mex(im2single(noisyI));

View Denoised Image

figure
imshowpair(noisyI,denoisedI,'montage');
title('Noisy Image (left) and Denoised Image (right)');

References

[1] Zhang, K., W. Zuo, Y. Chen, D. Meng, and L. Zhang. "Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising." IEEE Transactions on Image Processing. Vol. 26, Number 7, Feb. 2017, pp. 3142-3155.