About the Course
Parallelism has been employed for many years, mainly in high-performance computing. Interest in it has grown lately due to the physical constraints preventing frequency scaling. Parallel Computers can be roughly classified as either (i) shared-memory systems: multi-core/multi-processor computers having multiple processing elements in a single machine; or (ii) distributed-memory systems: typically using multiple computers to work on the same task. Typically, practical systems employ hybrid systems comprising of distributed memory systems with each computer comprising of shared memory system.
Distributed-memory systems are more scalable than shared-memory systems and are therefore more frequently used in Supercomputers. Such large scale systems typically comprise of a massive number of dedicated processors placed in close proximity to each other and connected using a special dedicated network such as a mesh or hypercube architecture. This saves considerable time in moving data around and makes it possible for the processors to work together. Supercomputers play an important role in computational science and are used in a wide range of fields, including weather forecasting, climate research, oil and gas exploration, and physical simulations.
Developing parallel programs involves aspects related to designing parallel algorithms, performing efficient communication and synchronization amongst the processors. In this course, we will be studying some basic computational routines that are used in Supercomputing (solving system of linear equations, fast Fourier transforms and graph problems). We will study algorithms for solving these problems in parallel and the underlying communication patterns. We will study various network topologies (mesh, hypercube, dragonfly) and relate the above communication patterns to these topologies.