Parallel Programming with MPI and Python

This is the course page for the parallel processing unit of the “Computational Economics Boot Camp.” This “camp” consisted of two courses, three credits each: Math 513R and Econ 413R, taught in 2013.

Welcome to the “Computational Economics Bootcamp” section on parallel programming! Below I have included outcome objectives, assignments & labs, class video lectures, and reference material.

Outcome Objectives—

Prerequisite Material

This material is a prerequisite to these labs.

  • comfortable with the unix command line evirnoment
    • learning the bash shell
    • ssh and scp (learn to use them efficiently)
    • compiling and running programs
    • the VI editor
  • Fulton Supercomputer
    • learn the SLURM scheduler on the Fulton Computers
    • submit standard serial jobs as well as parallel jobs

Parallel Programming

  • overview and essentials of parallel computing
    • Amdahl’s law
    • data parallel v. task parallel
  • parallel programming with MPI (taught with mpi4py)
    • intro to MPI
    • Point-to-Point communication
    • Collective Communication

Survey of existing parallel libraries and tools

    • brief overview of PETSc
    • overview of large data formats, such as HDF5

Dealing With Large Datasets

  • HDF5
    • Parallel and Normal HDF5
    • H5Py
    • Relational Datasets v HDF5

Assignments and Labs—

Assignment A (Prerequisite Assignment):

After you have obtained an account with the Fulton Supercomputer, watch the following instructional videos:

The first videos can be found on Youtube, through the BYU Supercomputing Channel and through my channel: 

On the BYU Supercomputing Channel, watch these videos:

On my channel, watch this video:

On Monday, we will talk about these and get everyone started on Fulton.

Fulton Job Submission Assignment

To further help you get up to speed, watch these two videos to demonstrate how to efficiently use ssh (through PuTTY and winSCP) and for some guidance on using the Unix command line.

Now complete the following mini-project. You will submit your first parallel job on the supercomputer: Job Submission Assignment

You will need the following file. Click to download: testJob

MPI Assignments:

At the end of the week, all exercises from the class text A Python Introduction to Parallel Programming with MPI will be due. This includes the chapters “An Overview of Parallel Computing,” “Introduction to MPI,” “Point-to-Point Communication,” “Collective Communication,” etc..

HDF Labs:

  • HDF Assignment 1 Here you will learn the purpose of the HDF and HDF5 formats and deleop some hand’s on experience with them. For this assignment, you will need this file: testHDF_AssignP1.zip

Note about Labs:

When turning in labs, you should submit them using standard Python documentation style (PEP 8 — Style Guide for Python). Good style and documentation is an important aspect of writing code and is a good habit to have. Too many times have I come back to my own code and not been able to remember exactly what it did and how it did it. Remember, “Code is read much more often than it is written.” Include many comments, plan out what your program will do ahead of time (so that it’s clear what your program takes in and what it returns), and make sure that it looks nice and readable.

Class Video Lectures—

Reference Material—

Class Text:

  • A Python Introduction to Parallel Programming with MPI.  A short introduction and tutorial to distributed memory computing with Python. Although HPC projects should usually be written in faster, compiled languages, like C or Fortran, Python is an excellent tool for algorithm development and proof of concept. This is especially true in parallel programming, where code becomes especially complex.

Using the Fulton Supercomputer:

Learning MPI: