1 / 15

INF5063 – GPU & CUDA

This recap provides a comprehensive overview of the CUDA programming model and the role of GPUs as coprocessors to CPUs. It explores the architecture of GPUs, including device memory, lightweight GPU threads, and the significance of parallel execution through kernels. Additionally, it outlines fundamental CUDA concepts such as grid, thread blocks, and memory access times. The document also discusses device memory allocation and host-device data transfer methods using `cudaMalloc` and `cudaMemcpy`, along with practical code examples for matrix operations and basic "Hello World" implementations in CUDA.

decker
Télécharger la présentation

INF5063 – GPU & CUDA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. INF5063 – GPU & CUDA Håkon Kvale Stensland iAD-lab, Department for Informatics

  2. Recap: The CUDA Programming Model • The GPU is viewed as a computedevicethat: • Is a coprocessor to the CPU, referred to as the host • Has its own DRAM called device memory • Runs manythreads in parallel • Data-parallel parts of an application are executed on the device as kernels, which run in parallel on many threads • Differences between GPU and CPU threads • GPU threads are extremely lightweight • Very little creation overhead • GPU needs 1000s of threads for full efficiency • Multi-core CPU needs only a few

  3. Recap: Terminology device = GPU = Set of multiprocessors Multiprocessor = Set of processors & shared memory Kernel = Program running on the GPU Grid = Array of thread blocks that execute a kernel Thread block = Group of SIMD threads that execute a kernel and can communicate via shared memory

  4. Recap: Access Times • Register – Dedicated HW – Single cycle • Shared Memory – Dedicated HW – Single cycle • Local Memory – DRAM, no cache – “Slow” • Global Memory – DRAM, no cache – “Slow” • Constant Memory – DRAM, cached, 1…10s…100s of cycles, depending on cache locality • Texture Memory – DRAM, cached, 1…10s…100s of cycles, depending on cache locality

  5. CUDA – API

  6. CUDA Highlights • The API is an extension to the ANSI C programming language Low learning curve than OpenGL/Direct3D • The hardware is designed to enable lightweight runtime and driver High performance

  7. (Device) Grid Block (0, 0) Block (1, 0) Shared Memory Shared Memory Registers Registers Registers Registers Thread (0, 0) Thread (1, 0) Thread (0, 0) Thread (1, 0) Local Memory Local Memory Local Memory Local Memory Host Global Memory Constant Memory Texture Memory CUDA Device Memory Allocation • cudaMalloc() • Allocates object in the device Global Memory • Requires two parameters • Address of a pointer to the allocated object • Size ofallocated object • cudaFree() • Frees object from device Global Memory • Pointer to the object

  8. CUDA Device Memory Allocation • Code example: • Allocate a 64 * 64 single precision float array • Attach the allocated storage to Md.elements • “d” is often used to indicate a device data structure BLOCK_SIZE = 64; Matrix Md int size = BLOCK_SIZE * BLOCK_SIZE * sizeof(float); cudaMalloc((void**)&Md.elements, size); cudaFree(Md.elements);

  9. (Device) Grid Block (0, 0) Block (1, 0) Shared Memory Shared Memory Registers Registers Registers Registers Thread (0, 0) Thread (1, 0) Thread (0, 0) Thread (1, 0) Local Memory Local Memory Local Memory Local Memory Host Global Memory Constant Memory Texture Memory CUDA Host-Device Data Transfer • cudaMemcpy() • memory data transfer • Requires four parameters • Pointer to source • Pointer to destination • Number of bytes copied • Type of transfer • Host to Host • Host to Device • Device to Host • Device to Device • Asynchronous operations available (Streams)

  10. CUDA Host-Device Data Transfer • Code example: • Transfer a 64 * 64 single precision float array • M is in host memory and Md is in device memory • cudaMemcpyHostToDevice and cudaMemcpyDeviceToHost are symbolic constants cudaMemcpy(Md.elements, M.elements, size, cudaMemcpyHostToDevice); cudaMemcpy(M.elements, Md.elements, size, cudaMemcpyDeviceToHost);

  11. CUDA Function Declarations • __global__ defines a kernel function • Must return void • __device__ and __host__ can be used together

  12. CUDA Function Declarations • __device__ functions cannot have their address taken • Limitations for functions executed on the device: • No recursion • No static variable declarations inside the function • No variable number of arguments

  13. Example: Hello World

  14. Example: Hello World // Hello World CUDA - INF5063 // #includetheentire body ofthecuPrintfcode (availible in the SDK) #include "util/cuPrintf.cu" #include <stdio.h> __global__ voiddevice_hello(void) { cuPrintf("Hello, world from the GPU!\n"); } intmain(void) { // greet from the CPU printf("Hello, world from the CPU!\n"); // initcuPrintf cudaPrintfInit(); // launch a kernelwith a single thread to say hi from thedevice device_hello<<<1,1>>>(); // display thedevice'sgreeting cudaPrintfDisplay(); // clean up aftercuPrintf cudaPrintfEnd(); return 0; }

  15. Example: Vector Addition

More Related