Tornado: A Practical And Efficient Heterogeneous Programming Framework For Managed Languages

This paper describes our experiences creating Tornado: a practical and efficient heterogeneous programming framework for managed languages. The novel aspect of Tornado is that it turns the programming of heterogeneous systems from an activity predominantly based on a priori knowledge into one based on a posteriori knowledge. Alternatively put, it simply means developers do not need to overcomplicate their code by catering for all possible eventualities. Instead, Tornado provides the ability to specialize each application for a specific system in situ which avoids the need for it to be pre-configured by the developer. To enable this, Tornado employs a sophisticated runtime system that can dynamically configure all aspects of the application - from selecting which parallelization scheme to apply to specifying which accelerators to use. By using this ability, the end-user, and not the developer, can transparently make use of any available multi-/many-core processor or hardware accelerator. To showcase the impact of Tornado, we implement a real-world computer vision application and deploy it across nine accelerators without having to modify the source code or even explicitly re-compile the application. Using dynamic configuration, we show that our implementation can achieve up to 124 frames per second (FPS) - up to 166x speedup over the reference implementation. Finally, our implementation is always within 21% of a hand-written OpenCL version but avoids much of the programming tedium.