Discriminating Equivalent Algorithms via Relative Performance

In scientific computing, it is common that a mathematical expression can be computed by many different algorithms (sometimes over hundreds), each identifying a specific sequence of library calls. Although mathematically equivalent, those algorithms might exhibit significant differences in terms of performance. However in practice, due to fluctuations, there is not one algorithm that consistently performs noticeably better than the rest. For this reason, with this work we aim to identify not the one best algorithm, but the subset of algorithms that are reliably faster than the rest. To this end, instead of using the usual approach of quantifying the performance of an algorithm in absolute terms, we present a measurement-based clustering approach to sort the algorithms into equivalence (or performance) classes using pair-wise comparisons. We show that this approach, based on relative performance, leads to robust identification of the fastest algorithms even under noisy system conditions. Furthermore, it enables the development of practical machine learning models for automatic algorithm selection.