updating materials

content/1-gpu-history.rst

@@ -10,7 +10,7 @@ Why GPUs?
 
 .. objectives::
 
-   - Explain the historical development of microprocessors and how GPUs enable 
+   - Explain the historical development of microprocessors and how GPUs enable
      continued scaling in computational power
 
 .. instructor-note::
@@ -19,7 +19,7 @@ Why GPUs?
    - 0 min exercises
 
 
-Moore's law
+The Moore's law
 -----------
 
 It states that the number of transistors in a dense integrated circuit doubles about every two years.
@@ -33,28 +33,28 @@ Higher performance of a single node has to rely on its more complicated structur
    The evolution of microprocessors.
    The number of transistors per chip increase roughly every 2 years.
    However, it can no longer be explored by the core frequency due to the power consumption limits.
-   Before 2000, the increase in the single core clock frequency was the major source of the 
+   Before 2000, the increase in the single core clock frequency was the major source of the
    increase in the performance. Mid 2000 mark a transition towards multi-core processors.
 
 Increasing performance has been sustained with two main strategies over the years:
 
-    - Increase the single processor performance: 
+    - Increase the single processor performance:
     - More recently, increase the number of physical cores.
 
 
 Computing in parallel
 ---------------------
 
-The underlying idea of parallel computing is to split a computational problem into smaller 
-subtasks. Many subtasks can then be solved *simultaneously* by multiple processing units. 
+The underlying idea of parallel computing is to split a computational problem into smaller
+subtasks. Many subtasks can then be solved *simultaneously* by multiple processing units.
 
-.. figure:: img/history/compp.png
+.. figure:: img/history/parallel-computing.png
    :align: center
-
-   Computing in parallel.
 
-How a problem is split into smaller subtasks strongly depends on the problem. 
-There are various paradigms and programming approaches to do this. 
+   Serial processing and parallel computing.
+
+How a problem is split into smaller subtasks strongly depends on the problem.
+There are various paradigms and programming approaches to do this.
 
 
 Graphics processing units
@@ -67,18 +67,18 @@ But over the years, they were used more and more in HPC.
 GPUs are a specialized parallel hardware for floating point operations.
 They are basically co-processors (helpers) for traditional CPUs: CPU still controls the work flow
 but it delegates highly-parallel tasks to the GPU.
-GPUs are based on highly parallel architectures, which allows taking advantage of the 
+GPUs are based on highly parallel architectures, which allows taking advantage of the
 increasing number of transistors.
 
 Using GPUs allows one to achieve extreme performance per node.
-As a result, the single GPU-equipped workstation can outperform small CPU-based clusters 
+As a result, the single GPU-equipped workstation can outperform small CPU-based clusters
 for some type of computational tasks. The drawback is: usually major rewrites of programs is required
 with an accompanying change in the programming paradigm.
 
 .. callout:: Host vs device
 
-   GPU-enabled systems require a heterogeneous programming model that involves both 
-   CPU and GPU, where the CPU and its memory are referred to as the host, 
+   GPU-enabled systems require a heterogeneous programming model that involves both
+   CPU and GPU, where the CPU and its memory are referred to as the host,
    and the GPU and its memory as the device.
 
 .. figure:: img/history/CPU_and_GPU_separated.png
@@ -92,7 +92,7 @@ A look at the Top-500 list
 
 The `TOP500 project <https://www.top500.org/>`__ ranks and details the 500 most powerful non-distributed computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year. The snapshot below shows the top-5 HPC systems as of June 2023, where the columns show:
 
-- **Cores** - Number of processors 
+- **Cores** - Number of processors
 - **Rmax** - Maximal LINPACK performance achieved
 - **Rpeak** - Theoretical peak performance
 - **Power** - Power consumption
@@ -116,13 +116,13 @@ GPU computing can significantly accelerate many types of scientific workloads.
 Improved energy efficiency
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Compared to CPUs, GPUs can perform more calculations per watt of power consumed, 
+Compared to CPUs, GPUs can perform more calculations per watt of power consumed,
 which can result in significant energy savings. This is indeed evident from the `Green500 list <https://www.top500.org/lists/green500/2023/06/>`__.
 
-Cost-effectiveness 
+Cost-effectiveness
 ^^^^^^^^^^^^^^^^^^
 
-GPUs can be more cost-effective than traditional CPU-based systems for certain 
+GPUs can be more cost-effective than traditional CPU-based systems for certain
 workloads.
 
 
@@ -132,17 +132,17 @@ Limitations and drawbacks
 Only for certain workloads
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Not all workloads can be efficiently parallelized and accelerated on GPUs. 
-Certain types of workloads, such as those with irregular data access patterns or 
+Not all workloads can be efficiently parallelized and accelerated on GPUs.
+Certain types of workloads, such as those with irregular data access patterns or
 high branching behavior, may not see significant performance improvements on GPUs.
 
 Steeper learning curve
 ^^^^^^^^^^^^^^^^^^^^^^
 
-Depending on the GPU programming API that you choose, GPU computing could 
-require specialized skills in GPU programming and knowledge of 
-GPU architecture, leading to a steeper learning curve compared to CPU programming. 
-Fortunately, if you study this training material closely you will become productive 
+Depending on the GPU programming API that you choose, GPU computing could
+require specialized skills in GPU programming and knowledge of
+GPU architecture, leading to a steeper learning curve compared to CPU programming.
+Fortunately, if you study this training material closely you will become productive
 with GPU programming quickly!