Instructor/Student Notes

Heterogenous computing models are increasing in importance in parallel and distributed computing. This module shows students a way to combine distributed computing techniques on clusters of machines with GPU computing on individual nodes in the cluster. The module contains background material and hands-on activities with partially-completed code segments and explanations of what students should try on their own to complete the task.

Prerequisites

Before getting started with this module, please keep in mind that we assume you have sufficient knowledge with C programming with Message Passing Interface(MPI) and CUDA C. If you are not familiar with MPI or CUDA C, please read and try the activities in the GPU Computing module, and the Distributed Computing Funcamentals module.

Intended Level

This set of background information and activities for students to try is intended for an intermediate or advanced level, once students are fairly comfortable with their programming skills, particularly using C. Because it build on two previous modules, it is likely that this module would end up as material for an advanced course, unless you intended to spend 3-4 weeks on parallel computing techniques in a computer organization course, for eample.

The code examples (addition of vectors and matrix-vector multiplication) follow from similar examples in the previous modules listed above, so they may provide a way for students to see how the same problem can be solved using different hardware architectures. Each example could be taught in serial versions first before having students work on these modules.

Cluster Resources

Instructors: The module called Distributed Computing Funcamentals contains information about cluster computing resources at Macalester College that we have used for this module. However, you will need to provide information about how to access the cluster you wish to use for your students to try the code.

Table Of Contents

Previous topic

Heterogeneous Computing Fundamentals

Next topic

Introduction to Heterogeneous Computing