Deploy pre-generated CUDA library functions to NVIDIA Jetson hardware

Good day,

I'm having some trouble figuring out how to configure GPU coder to compile and deploy a matlab function, which itself calls another CUDA function I have generated.

In other words, I have a function (cudaFoo.m):

function [c] = cudaFoo(a,b)
    c = a+b;
end

I have used GPU coder to compile that into a lib:

Now, in another function, parentFoo.m, I want to use that library function:

function [c] = parentFoo(a, b)
%#codegen

coder.cinclude('cudaFoo.h');

% Add include paths
coder.updateBuildInfo('addIncludePaths', 'codegen/lib/cudaFoo');

c = zeros(2,1);

% Call CUDA function: void cudaFoo(const double a[2], const double b[2], double c[2])
% This is a void function that modifies c as an output parameter - cannot be used as return value
coder.ceval('cudaFoo', coder.rref(a), coder.rref(b), coder.wref(c));

% Multiply result by 3
c = 3 * c;

end

I actually want parentFoo to run on my NVIDIA Jetson. So, I am trying to compile and deploy using the script below:

deviceAddress = '192.168.55.1'; %SSH IP address0
userName = 'jetson';
password = 'jetson';
hwobj = jetson(deviceAddress,userName,password);

cfg = coder.gpuConfig("exe");
cfg.Hardware = coder.hardware('NVIDIA Jetson');
cfg.Hardware.BuildDir = '~/matlab';
cfg.GenerateExampleMain = 'GenerateCodeAndCompile';
cfg.GenerateReport = true;
cfg.CustomSource = "cudaFoo.cu";
cfg.CustomInclude = "codegen\lib\cudaFoo";
cfg.CustomSourceCode = '#include "cudaFoo.h"';

inputArgs = {[1;2], [2;4]};
%codegen('-config ',cfg, 'parentFoo','-args',inputArgs);
codegen('-config', cfg, '-args', inputArgs, 'parentFoo', '-report');

Running the above gives the following error:

Error executing command "touch -c /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/*.*;make  -f parentFoo_rtw.mk -j4 all MATLAB_WORKSPACE="/home/jetson/matlab/MATLAB_ws/R2024b" LINUX_TARGET_LIBS_MACRO="" -C /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo". Details:

STDERR: /usr/bin/ld: cudaFoo.o: in function `cudaFoo(double const*, double const*, double*)':
tmpxft_0000873d_00000000-5_cudaFoo.cudafe1.cpp:(.text+0x14): undefined reference to `isInitialized_cudaFoo'
/usr/bin/ld: tmpxft_0000873d_00000000-5_cudaFoo.cudafe1.cpp:(.text+0x1c): undefined reference to `isInitialized_cudaFoo'
/usr/bin/ld: tmpxft_0000873d_00000000-5_cudaFoo.cudafe1.cpp:(.text+0x6c): undefined reference to `cudaFoo_initialize()'
collect2: error: ld returned 1 exit status
make: *** [parentFoo_rtw.mk:313: /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/parentFoo.elf] Error 1

STDOUT: make: Entering directory '/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo'
nvcc -rdc=true -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -c -Xcompiler -MMD,-MP -O2 -arch sm_50 -DMW_CUDA_ARCH=500 -DMW_GPU_MEMORY_MANAGER -D__MW_TARGET_USE_HARDWARE_RESOURCES_H__ -DMW_SCHED_OTHER=1  -D__linux__ -DARM_PROJECT -D_USE_TARGET_UDP_ -D_RUNONTARGETHARDWARE_BUILD_ -DSTACK_SIZE=200000 -DMODEL=parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox -I/home/jetson/matlab/MATLAB_ws/R2024b/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/examples -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/include -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/sources/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/toolbox/coder/rtiostream/src/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/extern/include -o MWMemoryManager.o /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/MWMemoryManager.cpp
nvcc -rdc=true -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -c -Xcompiler -MMD,-MP -O2 -arch sm_50 -DMW_CUDA_ARCH=500 -DMW_GPU_MEMORY_MANAGER -D__MW_TARGET_USE_HARDWARE_RESOURCES_H__ -DMW_SCHED_OTHER=1  -D__linux__ -DARM_PROJECT -D_USE_TARGET_UDP_ -D_RUNONTARGETHARDWARE_BUILD_ -DSTACK_SIZE=200000 -DMODEL=parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox -I/home/jetson/matlab/MATLAB_ws/R2024b/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/examples -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/include -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/sources/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/toolbox/coder/rtiostream/src/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/extern/include -o cudaFoo.o /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/lib/cudaFoo/cudaFoo.cu
nvcc -rdc=true -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -c -Xcompiler -MMD,-MP -O2 -arch sm_50 -DMW_CUDA_ARCH=500 -DMW_GPU_MEMORY_MANAGER -D__MW_TARGET_USE_HARDWARE_RESOURCES_H__ -DMW_SCHED_OTHER=1  -D__linux__ -DARM_PROJECT -D_USE_TARGET_UDP_ -D_RUNONTARGETHARDWARE_BUILD_ -DSTACK_SIZE=200000 -DMODEL=parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox -I/home/jetson/matlab/MATLAB_ws/R2024b/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/examples -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/include -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/sources/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/toolbox/coder/rtiostream/src/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/extern/include -o parentFoo_data.o /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/parentFoo_data.cu
nvcc -rdc=true -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -c -Xcompiler -MMD,-MP -O2 -arch sm_50 -DMW_CUDA_ARCH=500 -DMW_GPU_MEMORY_MANAGER -D__MW_TARGET_USE_HARDWARE_RESOURCES_H__ -DMW_SCHED_OTHER=1  -D__linux__ -DARM_PROJECT -D_USE_TARGET_UDP_ -D_RUNONTARGETHARDWARE_BUILD_ -DSTACK_SIZE=200000 -DMODEL=parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox -I/home/jetson/matlab/MATLAB_ws/R2024b/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/examples -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/include -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/sources/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/toolbox/coder/rtiostream/src/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/extern/include -o parentFoo_initialize.o /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/parentFoo_initialize.cu
nvcc -rdc=true -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -c -Xcompiler -MMD,-MP -O2 -arch sm_50 -DMW_CUDA_ARCH=500 -DMW_GPU_MEMORY_MANAGER -D__MW_TARGET_USE_HARDWARE_RESOURCES_H__ -DMW_SCHED_OTHER=1  -D__linux__ -DARM_PROJECT -D_USE_TARGET_UDP_ -D_RUNONTARGETHARDWARE_BUILD_ -DSTACK_SIZE=200000 -DMODEL=parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox -I/home/jetson/matlab/MATLAB_ws/R2024b/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/examples -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/include -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/sources/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/toolbox/coder/rtiostream/src/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/extern/include -o parentFoo_terminate.o /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/parentFoo_terminate.cu
nvcc -rdc=true -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -c -Xcompiler -MMD,-MP -O2 -arch sm_50 -DMW_CUDA_ARCH=500 -DMW_GPU_MEMORY_MANAGER -D__MW_TARGET_USE_HARDWARE_RESOURCES_H__ -DMW_SCHED_OTHER=1  -D__linux__ -DARM_PROJECT -D_USE_TARGET_UDP_ -D_RUNONTARGETHARDWARE_BUILD_ -DSTACK_SIZE=200000 -DMODEL=parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox -I/home/jetson/matlab/MATLAB_ws/R2024b/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/examples -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/include -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/sources/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/toolbox/coder/rtiostream/src/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/extern/include -o parentFoo.o /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/parentFoo.cu
nvcc -rdc=true -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -c -Xcompiler -MMD,-MP -O2 -arch sm_50 -DMW_CUDA_ARCH=500 -DMW_GPU_MEMORY_MANAGER -D__MW_TARGET_USE_HARDWARE_RESOURCES_H__ -DMW_SCHED_OTHER=1  -D__linux__ -DARM_PROJECT -D_USE_TARGET_UDP_ -D_RUNONTARGETHARDWARE_BUILD_ -DSTACK_SIZE=200000 -DMODEL=parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox -I/home/jetson/matlab/MATLAB_ws/R2024b/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/examples -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/include -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/sources/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/toolbox/coder/rtiostream/src/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/extern/include -o main.o /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/examples/main.cu
nvcc -rdc=true -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -c -Xcompiler -MMD,-MP -O2 -arch sm_50 -DMW_CUDA_ARCH=500 -DMW_GPU_MEMORY_MANAGER -D__MW_TARGET_USE_HARDWARE_RESOURCES_H__ -DMW_SCHED_OTHER=1  -D__linux__ -DARM_PROJECT -D_USE_TARGET_UDP_ -D_RUNONTARGETHARDWARE_BUILD_ -DSTACK_SIZE=200000 -DMODEL=parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox -I/home/jetson/matlab/MATLAB_ws/R2024b/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/lib/cudaFoo -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/examples -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/include -I/home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/sources/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/toolbox/coder/rtiostream/src/utils -I/home/jetson/matlab/MATLAB_ws/R2024b/C/Program_Files/MATLAB/R2024b/extern/include -o MW_nvidia_init.o /home/jetson/matlab/MATLAB_ws/R2024b/C/ProgramData/MATLAB/SupportPackages/R2024b/toolbox/target/supportpackages/nvidia/sources/utils/MW_nvidia_init.c
echo "### Creating standalone executable "/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/parentFoo.elf" ..."
### Creating standalone executable /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/parentFoo.elf ...
nvcc -lm -lrt -ldl -Xlinker -rpath,/usr/lib32 -Xnvlink -w -lcudart -lcuda -Wno-deprecated-gpu-targets -arch sm_50 -lcublas -lcusolver -lcufft -lcurand -lcusparse -o /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/parentFoo.elf MWMemoryManager.o cudaFoo.o parentFoo_data.o parentFoo_initialize.o parentFoo_terminate.o parentFoo.o main.o MW_nvidia_init.o  -lm -lcufft -lcublas -lcusolver -lm -lm
make: Leaving directory '/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo'

------------------------------------------------------------------------
Build error: C++ compiler produced errors. See the Build Log for further details.
More information
Code generation failed: View Error Report

Error using codegen

I.e., it seems to be missing the required dependencies. I've tried a few variations of the sourceFiles, etc., above, but, as a mechanical engineer, the file and dependency structure is all a bit of a closed book to me. Can anyone tell me what I am doing wrong in the configuration? Is this kind of nested GPU-coder-deployment possible?

Answer 1

[Chao Luo]

The library is missing for building the parentFoo exe. Also, you need to build the library on Jetson.

function c = parentFoo(a,b)
    coder.cinclude ('cudaFoo.h');
    coder.updateBuildInfo("addIncludePaths", '../../lib/cudaFoo');
    coder.updateBuildInfo("addLinkObjects", 'cudaFoo.a', '../../lib/cudaFoo');
    c = zeros(2,1);
    coder.ceval('cudaFoo', coder.rref(a), coder.rref(b), coder.wref(c));
    c = 3* c;
end

[Alyssa]

Hi @Chao Luo, thanks so much for your response.

Aha, I had assumed the library was transferred to the jetson automatically in my config. Clearly not! I have now sent it to the jetson using putFile, and successfully called the library functions using a small C++ test written directly on the nano.

However, I am still struggling to configure the include directories for parentFoo. I believe I have implemented your sugegstions. Here is my updated parentFoo.m:

function [c] = parentFoo(a, b)
%#codegen
    coder.cinclude('cudaFoo.h');
    % Add include paths
    coder.updateBuildInfo("addIncludePaths", '/home/jetson/matlab/MATLAB_ws/R2024b/Sandbox/cudaFooLib');
    coder.updateBuildInfo("addLinkObjects", 'cudaFoo.a', '/home/jetson/matlab/MATLAB_ws/R2024b/Sandbox/cudaFooLib');
    coder.updateBuildInfo('addSourcePaths','/home/jetson/matlab/MATLAB_ws/R2024b/Sandbox/cudaFooLib');
    
    c = zeros(2,1);
    coder.ceval('cudaFoo', coder.rref(a), coder.rref(b), coder.wref(c));
    c = 3 * c;
end

(As shown, I have stored the library in "/home/jetson/matlab/MATLAB_ws/R2024b/Sandbox/cudaFooLib", while the new .exe is theoretically destined for some other directory, at the moment: "/home/jetson/matlab/MATLAB_ws/R2024b/C/Users/....").

I can build the plain C++ source files (I guess since it isn't trying to run them), but the .exe fails:

STDERR: /home/jetson/matlab/MATLAB_ws/R2024b/C/Users/Alyssa/Documents/Sandbox/codegen/exe/parentFoo/parentFoo.cu:15:10: fatal error: cudaFoo.h: No such file or directory
 #include "cudaFoo.h"

So it still is not linked to the correct directory. I'm sure I'm missing something obvious, but what is it?

Thanks again for your help and patience.

[Chao Luo]

Because you are using absolute path, codegen on the hardware will prepend the workspace path(/home/jetson/matlab/MATLAB_ws/R2024b/) to them. So the header file path will be /home/jetson/matlab/MATLAB_ws/R2024b/home/jetson/matlab/MATLAB_ws/R2024b/Sandbox/cudaFooLib, and it does not exist.

Using relative path would be easier and it would work on both host and the hardware target.