Beyond Pass/Fail: The Story of Learning-Based Testing

Learning-Based Testing (LBT) merges learning and testing processes to achieve both testing and behavioral adequacy. LBT utilizes active learning to infer the model of the System Under Test (SUT), enabling scalability for large and complex programs by requiring only a minimal set of initial test cases. The core principle of LBT is that the SUT's behavior can be thoroughly inferred by progressively generating test cases and subjecting the SUT to testing, thereby ensuring comprehensive testing. Despite being in its early stages, LBT has a solid foundation of theoretical research demonstrating its efficacy in testing both procedural and reactive programs. This paper provides a systematic literature review of various LBT implementations across different program types and evaluates the current state of research in this field. We explore diverse theoretical frameworks, existing tools, and libraries within the LBT domain to illustrate the concept's evolution and current research status. Additionally, we examine case studies involving the application of LBT tools in industrial settings, highlighting their potential and effectiveness in commercial software testing. This systematic literature review aims to offer researchers a comprehensive perspective on the inception and development of LBT, presenting it as a promising technique in software testing. By unveiling LBT's underutilized potential, this paper seeks to significantly benefit the practitioners and research community.