A Multi-level Methodology for Behavioral Comparison of Software-Intensive Systems

Software-intensive systems constantly evolve. To prevent software changes from unintentionally introducing costly system defects, it is important to understand their impact to reduce risk. However, it is in practice nearly impossible to foresee the full impact of software changes when dealing with huge industrial systems with many configurations and usage scenarios. To assist developers with change impact analysis we introduce a novel multi-level methodology for behavioral comparison of software-intensive systems. Our fully automated methodology is based on comparing state machine models of software behavior. We combine existing complementary comparison methods into a novel approach, guiding users step by step though relevant differences by gradually zooming into more and more detail. We empirically evaluate our work through a qualitative exploratory field study, showing its practical value using multiple case studies at ASML, a leading company in developing lithography systems. Our method shows great potential for preventing regressions in system behavior for software changes.