GraphFuzz: Automated Testing of Graph Algorithm Implementations with Differential Fuzzing and Lightweight Feedback

Graph algorithms, such as shortest path finding, play a crucial role in enabling essential applications and services like infrastructure planning and navigation, making their correctness important. However, thoroughly testing graph algorithm implementations poses several challenges, including their vast input space (i.e., arbitrary graphs). Moreover, through our preliminary study, we find that just a few automatically generated graphs (less than 10) could be enough to cover the code of many graph algorithm implementations, rendering the code coverage-guided fuzzing approach -- one of the state-of-the-art search algorithms -- less efficient than expected. To tackle these challenges, we introduce GraphFuzz, the first automated feedback-guided fuzzing framework for graph algorithm implementations. Our key innovation lies in identifying lightweight and algorithm-specific feedback signals to combine with or completely replace the code coverage feedback to enhance the diversity of the test corpus, thereby speeding up the bug-finding process. This novel idea also allows GraphFuzz to effectively work in both black-box (i.e., no code coverage instrumentation/collection is required) and grey-box setups. GraphFuzz applies differential testing to detect both crash-triggering bugs and logic bugs. Our evaluation demonstrates the effectiveness of GraphFuzz. The tool has successfully discovered 12 previously unknown bugs, including 6 logic bugs, in 9 graph algorithm implementations in two popular graph libraries, NetworkX and iGraph. All of them have been confirmed and and 11 bugs have been rectified by the libraries' maintainers.