Expressing Parallelism


Unless you live in a cave, you’re aware of the microarchitectural paradigm shift towards multicore/multinode technology. This computational sea change is accompanied by a heated debate about how programming languages should express parallelism. Graduate students around the world invent parallel programming languages du jour, but the most popular solutions for expressing parallelism are OpenMP and MPI. Despite their widespread appeal, both of these packages suffer unique limitations. Although OpenMP provides an expedient avenue for parallelizing sequential code, OpenMP can only be implemented on shared-memory architectures. On the other hand, MPI is extensible to a diversity of architectures, but requires verbose expression of boundary conditions and inter-node communication.

In response to these language limitations, I point to the ZPL programming language. ZPL is developed at the University of Washington and demonstrably proves two concepts:

  1. R aising the semantic level of a programming language eliminates the problem of coalescing communication (i.e. – bundling/scheduling concurrent inter-node messages).
  2. A sophisticated high-level language can eliminate the need to express boundary conditions.

Most importantly, ZPL accomplishes #1 and #2 with little (or no) performance loss, as compared to languages like High Performance Fortran. How is ZPL so wonderful? Read “The Design and Development of ZPL”. In short, ZPL introduces a concept called regions, which allows data dependencies to be described independently of the processor-to-data mapping. (The aforementioned paper also discusses several more innovations in ZPL.)

So, what’s my point? When we write parallel code, let’s not complacently use ancient tools.  Although OpenMP and MPI can be used effectively in some contexts, we should also embrace next-generation languages.  ZPL allows programmers to globally orchestrate parallel algorithms, which is certainly worth the learning-curve. Ultimately, the use of next-gen languages promotes the improvement of said languages.


One Response to “Expressing Parallelism”

  1. Dr. Iglinksi Says:

    I hate to make you look uninformed on your public blog, but while you have been in graduate school utopia, I have been writing extensions to MS-BASIC that will do all this and more with a 10% efficiency gain over ZPL on Motorola 6800-family architectures. IPO coming soon.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: