Deep learning and the hardware that has evolved to support it have shifted the focus of distributed computing back to the approaches popular when supercomputers were the primary environment for large scale coordinated computing. This stands in contrast to the primary framework for distributed computing that has dominated the past decade with its focus on distributing compute across the data center. 
The talk covers some of the lessons from supercomputing systems that are once again relevant in a world where deep learning is increasing effective in tackling interesting problems.