Broadcast

There is often a need for all the processes to share data by ensuring that all of them have a copy of the same data in the memory of their process. This form of communication is often done with a broadcast from the master to all of the worker processes.

08. Broadcast: a special form of message passing

file: patternlets/MPI/08.broadcast/broadcast.c

Build inside 08.broadcast directory:

make broadcast

Execute on the command line inside 08.broadcast directory:

mpirun -np <number of processes> ./broadcast

This example shows how a data item read from a file can be sent to all the processes. Lines 29 through 34 demonstrate reading data from a file. After opening the file and asserting that the file is not empty, the file is read by the fscanf function. This function then stores the data from the file as an integer in the answer variable. Note that only process 0 has the data from the file stored in answer.

In order to send the data from process 0 to all of the processes in the communicator, it is necessary to broadcast. During a broadcast, one process sends the same data to all of the processes. A common use of broadcasting is to send user input to all of the processes in a parallel program. In our example, the broadcast is sent from process 0 and looks like this:

/* broadcast.c
 * ... illustrates the use of MPI_Bcast() with a scalar value...
 *      (compare to array version).
 * Joel Adams, Calvin College, April 2016.
 *
 * Usage: mpirun -np N ./broadcast
 *
 * Exercise:
 * - Compile and run several times,
 *     using 2, 4, and 8 processes
 * - Use source code to trace execution and output
 *     (noting contents of file "data.txt");
 * - Explain behavior/effect of MPI_Bcast().
 */

#include <mpi.h>
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>

int main(int argc, char** argv) {
	int answer = 0;
	int numProcs = 0, myRank = 0;

	MPI_Init(&argc, &argv);
	MPI_Comm_size(MPI_COMM_WORLD, &numProcs);
	MPI_Comm_rank(MPI_COMM_WORLD, &myRank);

	if (myRank == 0) {
		FILE *filePtr = fopen("data.txt", "r");
		assert( filePtr != NULL );
		fscanf(filePtr, " %d", &answer);
		fclose(filePtr);
	}

	printf("BEFORE the broadcast, process %d's answer = %d\n",
	myRank, answer);

	MPI_Bcast(&answer, 1, MPI_INT, 0, MPI_COMM_WORLD);

	printf("AFTER the broadcast, process %d's answer = %d\n",
	myRank, answer);

	MPI_Finalize();

	return 0;
}

Note

In this single-program, multiple data scenario, all processes execute the MPI_Bcast function. The fourth parameter dictates which process id is doing the sending to all of the other processes, who are waiting to receive.

For further exploration

This program illustrates a common pattern in many parallel and distributed programs: the master node (very typically rank 0) is the only one to read in data from a file. It is then responsible for distributing the data to all of the other processes. MPI also has defined collective operations for file reading and writing that can be done in parallel. This is referred to as parallel IO. You could investigate this topic further. Your system will need to support it to take advantage of it.

To do:

Run this code with several processes (8 - 12). What do you observe about the order in which the printing occurs among the processes? Is it repeatable or does it change each time you run it?

09. Broadcast: incorporating user input

file: patternlets/MPI/09.broadcastUserInput/broadcastUserInput.c

Build inside 09.broadcastUserInput directory:

make broadcastUserInput

Execute on the command line inside 09.broadcastUserInput directory:

mpirun -np <number of processes> ./broadcastUserInput <integer>

We can use command line arguments to incorporate user input into a program. Command line arguments are taken care of by two functions in main(). The first of these functions is argc which is an integer referring to the number of arguments passed in on the command line. argv is the second function. It is an array of pointers that points to each argument passed in. argv[0] always holds the name of the program and in MPI argv[1] holds the number of processes when initially run using mpirun. However, after the MPI_Init function is called as in line 51 below, argc and argv are updated to contain only the command line arguments to the program itself.

We modified the previous broadcast example to include an additional command line argument, an integer. Instead of reading a scalar value from a file, this allows a user to decide what value is broadcast in the program when it is executed. In this example, after line 51, argc is 2, and argv[1] contains the integer you entered at the end of the command. The function called getInput then acts like any other C program with command line arguments.

To do:

Try running the program without an argument for the number of processes.

mpirun ./broadcastUserInput <integer>

What is the default number of processes used on your system when we do not provide a number?

/* broadcastUserInput.c
 * ... illustrates the use of MPI_Bcast() with a scalar value
 *     obtained via a command line argument.
 *
 * Hannah Sonsalla, Macalester College 2017
 * Modeled from code by Joel Adams, Calvin College, April 2016.
 *
 * Usage: mpirun -np N ./broadcastUserInput <integer>
 *
 * Exercise:
 * - Compile and run several times varying the number
 *   of processes and integer value
 * - Explain the behavior you observe
 */

#include <mpi.h>
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>

#define MASTER 0

/* gets value of answer from user
 * @param: argc, argument count.
 * @param: argv, argument pointer array.
 * @param: myRank, rank of current process
 * @param: answer, variable to store value given by user
 * Precondition: argc is a count of the number of arguments.
 *              && argv is a pointer array that points to the arguments.
 *              && myRank is the rank of this MPI process.
 *		&& answer is the variable to be assigned value.
 * Postcondition: answer has been filled with value from user
 *                if given, else answer remains set to 0.
 */
void getInput(int argc, char* argv[], int myRank, int* answer) {

    if (myRank == 0){  // master process
        if (argc == 2){
             *answer = atoi(argv[1]);
        }
    }
    MPI_Bcast(answer, 1, MPI_INT, 0, MPI_COMM_WORLD);
}

int main(int argc, char** argv) {
    int answer = 0, length = 0;
    int myRank = 0;

    char myHostName[MPI_MAX_PROCESSOR_NAME];

    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &myRank);
    MPI_Get_processor_name (myHostName, &length);

    printf("BEFORE the broadcast, process %d on host '%s' has answer = %d\n",
             myRank, myHostName, answer);

    getInput(argc, argv, myRank, &answer);

    printf("AFTER the broadcast, process %d on host '%s' has answer = %d\n",
             myRank, myHostName, answer);

    MPI_Finalize();

    return 0;
}

10. Broadcast: send receive equivalent

file: patternlets/MPI/10.broadcastSendReceive/broadcastSendReceive.c*

Build inside 10.broadcastSendReceive directory:

make broadcastSendReceive

Execute on the command line inside 10.broadcastSendReceive directory:

mpirun -np <number of processes> ./broadcastSendReceive

This example shows how to ensure that all processes have a copy of an array created by a single master process. Master process 0 sends the array to each process, all of which receive the array.

Note

This code works to broadcast data to all processes, but this is such a common pattern that the MPI library contains the built-in MPI_Bcast function. Compare this example to the following one to see the value in using the single function.

/*
 * broadcastSendReceive.c
 * ... illustrates basic send receive functions.
 * Master process sends filled array to each process.
 *
 * Hannah Sonsalla, Macalester College 2017
 * fill and print function from code by Joel Adams, Calvin College
 *
 * Usage: mpirun -np N ./broadcastSendReceive
 *
 * Exercise:
 * - Compile and run, using 2, 4, and 8 processes
 * - Use source code to trace execution and output
 * 
 */

#include <mpi.h>
#include <stdio.h>
#include <stdlib.h>

/* fill an array with some arbitrary values 
 * @param: a, an int*.
 * @param: size, an int.
 * Precondition: a is the address of an array of ints.
 *              && size is the number of ints a can hold.
 * Postcondition: a has been filled with arbitrary values 
 *                { 11, 12, 13, ... }.
 */
void fill(int* a, int size) {
	int i;
	for (i = 0; i < size; i++) {
		a[i] = i+11;
	}
}

/* display a string, a process id, and its array values 
 * @param: str, a char*
 * @param: id, an int
 * @param: a, an int*.
 * Precondition: str points to either "BEFORE" or "AFTER"
 *              && id is the rank of this MPI process
 *              && a is the address of an 8-element int array.
 * Postcondition: str, id, and a have all been written to stdout.
 */
void print(char* str, int id, int* a) {
	printf("%s array sent, process %d has: {%d, %d, %d, %d, %d, %d, %d, %d}\n",
	   str, id, a[0], a[1], a[2], a[3], a[4], a[5], a[6], a[7]);
}

#define MAX 8

int main(int argc, char** argv) {
	int id = -1, numProcesses = -1;
	int array[MAX] = {0};
    

	MPI_Init(&argc, &argv);
	MPI_Comm_rank(MPI_COMM_WORLD, &id);
    	MPI_Comm_size(MPI_COMM_WORLD, &numProcesses);
    
	if (id == 0) fill(array, MAX);
     
	print("BEFORE", id, array);
	
	// master process sends array to every process
	if (id == 0) {
		for (int i = 1; i < numProcesses; i++) {
			MPI_Send(&array, MAX, MPI_INT, 
			    i, 1, MPI_COMM_WORLD);
	    }
	}
	
	else {
	    MPI_Recv(&array, MAX, MPI_INT, 0, 
	        1, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
	}
	
    	print("AFTER", id, array);
 	MPI_Finalize();

	return 0;
}

11. Broadcast: send data to all processes

file: patternlets/MPI/11.broadcast2/broadcast2.c

Build inside 11.broadcast2 directory:

make broadcast2

Execute on the command line inside 11.broadcast2 directory:

mpirun -np <number of processes> ./broadcast2

The send and receive pattern where one process sends the same data to all processes is used frequently. Broadcast was created for this purpose. This example is the same as the previous example except that we send the modified array using broadcast.

/* broadcast2.c
 * ... illustrates the use of MPI_Bcast() for arrays...
 * Joel Adams, Calvin College, November 2009.
 *
 * Usage: mpirun -np N ./broadcast2
 *
 * Exercise:
 * - Compile and run, using 2, 4, and 8 processes
 * - Use source code to trace execution and output
 * - Explain behavior/effect of MPI_Bcast().
 */

#include <mpi.h>
#include <stdio.h>
#include <stdlib.h>

/* fill an array with some arbitrary values
 * @param: a, an int*.
 * @param: size, an int.
 * Precondition: a is the address of an array of ints.
 *              && size is the number of ints a can hold.
 * Postcondition: a has been filled with arbitrary values
 *                { 11, 12, 13, ... }.
 */
void fill(int* a, int size) {
	int i;
	for (i = 0; i < size; i++) {
		a[i] = i+11;
	}
}

/* display a string, a process id, and its array values
 * @param: str, a char*
 * @param: id, an int
 * @param: a, an int*.
 * Precondition: str points to either "BEFORE" or "AFTER"
 *              && id is the rank of this MPI process
 *              && a is the address of an 8-element int array.
 * Postcondition: str, id, and a have all been written to stdout.
 */
void print(char* str, int id, int* a) {
	printf("%s broadcast, process %d has: {%d, %d, %d, %d, %d, %d, %d, %d}\n",
	   str, id, a[0], a[1], a[2], a[3], a[4], a[5], a[6], a[7]);
}

#define MAX 8

int main(int argc, char** argv) {
	int array[MAX] = {0};
	int numProcs, myRank;

	MPI_Init(&argc, &argv);
	MPI_Comm_size(MPI_COMM_WORLD, &numProcs);
	MPI_Comm_rank(MPI_COMM_WORLD, &myRank);

	if (myRank == 0) fill(array, MAX);

	print("BEFORE", myRank, array);

	MPI_Bcast(array, MAX, MPI_INT, 0, MPI_COMM_WORLD);

	print("AFTER", myRank, array);

	MPI_Finalize();

	return 0;
}