Parallel Computing Using Cuda-C

This repository contains code examples and resources for parallel computing using CUDA-C. CUDA-C is a parallel computing platform and programming model developed by NVIDIA, specifically designed for creating GPU-accelerated applications.

The goal of this repository is to provide beginners with a starting point to understand parallel computing concepts and how to utilize CUDA-C to leverage the power of GPUs for accelerating computationally intensive tasks. Whether you are a student, researcher, or developer interested in parallel computing, this repository aims to provide a practical guide and code examples to get you started.

Introduction to CUDA-C

CUDA-C is an extension of the C programming language that allows developers to write code that can be executed on NVIDIA GPUs. It provides a set of language extensions, libraries, and tools that enable developers to harness the power of parallel processing on GPUs.

CUDA-C allows you to write parallel code using the CUDA programming model, which includes defining kernels (functions that execute on the GPU) and managing data transfers between the CPU and GPU. By writing CUDA-C code, you can achieve significant speedups for computationally intensive tasks compared to running the same code on the CPU alone.

Why we need Cuda-C

With the exponential growth of data and increasing demands from users, CPUs alone are no longer sufficient for efficient processing. GPUs offer parallel processing capabilities, making them well-suited for handling large-scale computations. CUDA-C, developed by NVIDIA, enables developers to leverage GPUs for accelerated processing, resulting in faster and more efficient data processing.

Getting Started

If your computer has GPU

Following these steps in NIVIDA to install Cuda Toolkit

• If you are using Linux, I advise you to watch this video

• If you are using Windows, this is your video

If your computer doesn't have GPU

• Don't worry; I'll demonstrate how to set up and use Google Colab to code in here

Table of Contents

• Chapter01: Demystifying CPUs and GPUs: What You Need to Know

• Chapter02: How the way a computer works

• Chapter03: Terminology in parallel programming

• Chapter04: Hello world Cuda-C

• Chapter05: The operational mechanism of CPU-GPU

• Chapter06: Memory Types in GPU

• Chapter07: Using GPU memory

• Chapter08: Synchronization and Asynchronization

• Chapter09: Unified memory

• Chapter10: Pinned memory

• Chapter11: Streaming

• Chapter12: Data Hazard

• Chapter13: Atomic Function

• Chapter14: Shared memory

Resources

In addition to the code examples, this repository provides a curated list of resources, including books, tutorials, online courses, and research papers, to further enhance your understanding of parallel computing and CUDA-C programming. These resources will help you delve deeper into the subject and explore advanced topics and techniques.

• NVIDIA Practices_Guide 2023
• NVIDIA Programming_Guide 2023
• GPU Programming 2021 in youtube
• Cuda Programming 2023 in youtube
• Programming Massively 2022 in youtube
• Cuda training series 2022-2023 in youtube
• Programming Heterogeneous Computing Systems with GPUs 2023 in youtube
• Cuda Thread Indexing cheatsheet