Talk by Yishai Oltchik (Hebrew University of Jerusalem): Network Topologies and Inevitable Contention

16.10.2017 11:00

CAB E 72

Talk by Yishai Oltchik (Hebrew University of Jerusalem)

Title: Network Topologies and Inevitable Contention


Network topologies can have significant effect on the execution costs of parallel algorithms due to inter-processor communication. For particular combinations of computations and network topologies, costly network contention may inevitably become a bottleneck, even if algorithms are optimally designed so that each processor communicates as little as possible. We obtain novel contention lower bounds that are functions of the network and the computation graph parameters. For several combinations of fundamental computations and common network topologies, our new analysis improves upon previous per-processor lower bounds which only specify the number of words communicated by the busiest individual processor. We consider torus and mesh topologies, universal fat-trees, and hypercubes; algorithms covered include classical matrix multiplication and direct numerical linear algebra, fast matrix multiplication algorithms, programs that reference arrays, N-body computations, and the FFT. For example, we show that fast matrix multiplication algorithms (e.g., Strassen’s) running on a 3D torus will suffer from contention bottlenecks. On the other hand, this network is likely sufficient for a classical matrix multiplication algorithm. Our new lower bounds are matched by existing algorithms only in very few cases, leaving many open problems for network and algorithmic design.


Yishai Oltchik is currently a MSc student at the Hebrew University of Jerusalem under the supervision of Oded Schwartz, researching parallel computation and HPC. His primary research interests are in the fields of network topologies and parallel algorithms.