Getting Started With CUDA C on an Nvidia Jetson: GPU Architecture

Mohammed Billoo

Still RelevantIntermediate

In the previous blog post (Getting Started With CUDA C on Jetson Nvidia: Hello CUDA World!) I showed how to develop applications targeted at a GPU on a Nvidia Jetson Nano. As we observed in that blog post, performing a calculation on a 1-D array on a GPU had no performance benefit compared to a traditional CPU implementation, even on an array with many elements. In this blog post, we will learn about the GPU architecture to better explain the behavior and to understand the applications where a GPU shines (hint: it has to do with graphics).

Summary

This blog post explains the Nvidia Jetson GPU architecture to clarify why simple 1-D array CUDA examples often show no speedup and to show where GPUs provide real benefits. Readers will learn how streaming multiprocessors, warps, and the CUDA memory hierarchy map to CUDA C and affect performance on embedded Linux Jetson platforms.

Key Takeaways

Explain the Jetson GPU execution model: streaming multiprocessors (SMs), warps, SIMT execution, and how CUDA threads map to hardware.
Identify the types of workloads that benefit from GPU acceleration versus CPU execution, and why trivial 1-D array tasks may not.
Apply CUDA performance concepts—occupancy, memory coalescing, shared vs. global memory, and kernel launch configuration—to improve throughput.
Use profiling and optimization strategies (Nsight, minimizing host-device copies, maximizing parallelism) to measure and tune CUDA apps on Jetson.

Who Should Read This

Embedded engineers or software developers with C/C++ experience who want to understand how GPU architecture impacts CUDA C performance on Nvidia Jetson boards running embedded Linux.

Still RelevantIntermediate