3.2. Parallel for Loop Program Structure: chunks of 1¶
Program file: 03parallelLoopChunksOf1.py
Example usage:
python run.py ./03parallelLoopChunksOf1.py N
Here the N signifies the number of processes to start up in mpi.
run.py executes this program within mpirun using the number of processes given.
Exercises:
Run, using these numbers of processes, N: 1, 2, 4, and 8 (i.e., vary the last argument to run.py).
Compare source code to output.
Change REPS to 16, save, rerun, varying N again.
Explain how this pattern divides the iterations of the loop among the processes.
- The processes are assigned single iterations in order of their process number.
- The processes are assigned single iterations in order of their process number.
- In this code, single iterations, or chunks of size 1 repetition are assigned consecutively to each process.
Q-1: Which of the following is the correct assignment of loop iterations to processes for this code, when REPS is 8 and numProcesses is 4?
3.2.1. Dive into the code¶
In the code below, we again use the variable called REPS for the total amount or work, or repetitions, that the for loop is accomplishing. This particular code is designed so that the number of repetitions should be more than or equal to the number of processes requested. .. note:: Typically in real problems, the number of repetitions is much higher than the number of processes. We keep it small here to illustrate what is happening.
Like the last example all processes execute the code in the part of the if statement that evaluates to True. Note that in the for loop in this case we simply have process whose id is 0 start at iteration 0, then skip to 0 + numProcesses for its next iteration, and so on. Similarly, process 1 starts at iteration 1, skipping next to 1+ numProcesses, and continuing until REPs is reached. Each process performs similar single ‘slices’ or ‘chunks of size 1’ of the whole loop.
from mpi4py import MPI
def main():
comm = MPI.COMM_WORLD
id = comm.Get_rank() #number of the process running the code
numProcesses = comm.Get_size() #total number of processes running
myHostName = MPI.Get_processor_name() #machine name running the code
REPS = 8
if (numProcesses <= REPS):
for i in range(id, REPS, numProcesses):
print("On {}: Process {} is performing iteration {}"\
.format(myHostName, id, i))
else:
# can't hove more processes than work; one process reports the error
if id == 0 :
print("Please run with number of processes less than \
or equal to {}.".format(REPS))
########## Run the main function
main()