Training a deep CNN

Which data layout (NHWC, NCHW, or CHWN) is used in trainig a deep CNN? Is there a possibility to choose one of them for the training process?

Answer 1

[Krishna]

Hello Ali,

In deep Convolutional Neural Networks (CNNs) training, the NCHW and NHWC data formats are widely utilized, with the choice between them influenced by the deep learning framework, hardware (GPUs or CPUs), and model or optimization needs.

NCHW: Number of samples, Channels, Height, Width. Preferred for NVIDIA GPUs due to optimization for CUDA cores.

NHWC: Number of samples, Height, Width, Channels. Often the default in TensorFlow, this intuitive format is favored for CPU computations or with specific accelerators.

CHWN: Less common and rarely supported in major frameworks like TensorFlow or PyTorch.

Choosing a Data Layout involves considering framework support, hardware compatibility (with NVIDIA GPUs favoring NCHW), model requirements, and performance impacts.

Please go through the following article to learn more,

https://medium.com/@deepika_writes/nhwc-vs-nchw-a-memory-access-perspective-on-gpus-4e79bd3b1b54

Hope this helps.