Evolution of a Parallel Program

Scott Burlington

Introduction

This tutorial will document the evolution of a self-contained serial program to a program intended to run in a high-performance multi-processor, multi-node computing environment. In particular we would like to illustrate how to take a piece of serial, computationally intensive segment of code and have it run effectively on our beowulf cluster.

The differences from a standard serial program will have to address the target hardware environment that we wish to utilize. These considerations will span the use of multiple processors in a shared memory environment (SMP - shared memory processor) as well as the use of a network of tightly coupled identical machines sharing a common file system in a distributed memory environment. This second concept is sometimes referred to loosely as a Network of Workstations (NOW) although this term is generally depricated in its application to what is now widely known as a Beowulf Cluster.

To achieve this we will document the following evolution:

A program written in the C programming language
A multi-threaded C program using posix compliant P-threads
A distributed version running on a Parallel Virtual Machine (PVM)

Considerations regarding shared memory space, data sharing and load balancing will be noted in particular. Decisions and explainations with code snippits and complete code examples will be given. As well as some notes on the usage of the pre-installed software used.