In the previous post, I explained the necessity for novel parallel programming languages. In this post, I will discuss the precision of parallel languages. By precision, I mean the degree to which an algorithm’s parallel implementation can be optimized for a particular computational architecture.
In 1982, the Poker programming environment introduced three semantic levels of parallel code. The lowest level of code focuses on the execution of a single processor within a larger multiprocessor system. At the middle level, code describes the communication protocol between two or more processors. Finally, the highest level of code orchestrates a global algorithm.
Poker’s three semantic levels provide a good framework to discuss the precision of parallel languages. Using conventional parlance, languages can be explicitly parallel or implicitly parallel. Explicit languages emphasize the lowest and middle levels of the Poker model. Explicit languages verbosely express inter-process communication and robustly articulate the mapping of data to processors. The explicit focus on lower semantic levels typically obfuscates any cogency of the overall parallel algorithm. On the other hand, implicit languages emphasize a higher semantic level and express parallelism in an abstract fashion. Although implicit languages provide semantic terseness, they traditionally incur an additional performance overhead because they cannot be fine-tuned for a particular architecture.
Explicit and implicit languages assume two different philosophies for achieving computational performance. Explicit languages rely on human developers to effect fine-grain architecture-specific adjustments to a code’s performance. Implicit languages rely on sophisticated compilers to “auto-magically” optimize data mappings and inter-process communication. Until recently, it was assumed that human optimizations outperformed compiler optimizations. In other words, the following relationship was historically true:
Explicit languages: verbose semantics => fine-tuned performance
Implicit languages: clean semantics => suboptimal performance
Today I listened to Qing Yi present a talk titled “Parameterizing optimizations for emprical tuning (POET)”. POET scripts provide language-independent annotations which can be inserted into existing code. The POET annotations can be written automatically by a compiler or manually by a human. POET’s philosophy is to allow developers to get at the results of the compiler (in a more explicit manner than compiler flags). Using this paradigm, it should be easy to envision a hybridization of explicit and implicit languages. In the ideal future, we could use highly abstract languages to implement parallel algorithms, and intermediary tools (such as POET) to fine-tune the compilation.
So what’s my point? Developers typically assume that compilers cannot generate efficient parallel code. Consequently, developers accept the burden of explicitly expressing parallelism. Emergent compiler tools, such as POET, point towards a hybridized future where we can enjoy the semantic ease of implicitly parallel languages with the fine-grained optimizations of explicit languages.