Distributed Computing Fundamentals Using Python


This book contains some examples illustrating the basic fundamental concepts of distributed computing using Python code. The type of computing these examples illustrate is called message passing. Message passing is a form of programming that is based on processes that communicate with each other to coordinate their work. Message passing can be used on a single multicore computer or with a cluster of computers.

Software Patterns

Patterns in software are common implementations that have been used over and over by practitioners to accomplish tasks. As practitioners use them repeatedly, the community begins to give them names and catalog them, often turning them into reusable library functions. The examples you will see in this book are based on documented patterns that have been used to solve different problems using message passing between processes. Message passing is one form of distributed computing using processes, which can be used on clusters of computers or multicore machines.

In many of these examples, the pattern’s name is part of the python code file’s name. You will also see that often the MPI library functions also take on the name of the pattern, and the implementation of those functions themselves contains the pattern that practitioners found themselves using often. These pattern code examples we show you here, dubbed patternlets, are based on original work by Joel Adams (4).

Software Details

These examples use a package called mpi4py (1, 2, 3). The mpi4py package relies on an underlying C code implementation of a standard called Message Passing Interface (MPI). The mpi4py package function names have direct mappings to the underlying MPI C library function names. MPI and its implementations have been around a long time and have stood the test of time. It is useful to understand the principles of distributed computing using this standard.


  1. Dalcin, P. Kler, R. Paz, and A. Cosimo, Parallel Distributed Computing using Python, Advances in Water Resources, 34(9):1124-1139, 2011. http://dx.doi.org/10.1016/j.advwatres.2011.04.013

  1. Dalcin, R. Paz, M. Storti, and J. D’Elia, MPI for Python: performance improvements and MPI-2 extensions, Journal of Parallel and Distributed Computing, 68(5):655-662, 2008. http://dx.doi.org/10.1016/j.jpdc.2007.09.005

  1. Dalcin, R. Paz, and M. Storti, MPI for Python, Journal of Parallel and Distributed Computing, 65(9):1108-1115, 2005. http://dx.doi.org/10.1016/j.jpdc.2005.03.010


Adams, Joel C. “Patternlets: A Teaching Tool for Introducing Students to Parallel Design Patterns.” 2015 IEEE International Parallel and Distributed Processing Symposium Workshop. IEEE, 2015.

You have attempted of activities on this page
Next Section - 1. What you will need