Challenges in Comparing Code Maintainability across Different Programming Languages

Comparing the quality of software written in different computer languages is required in a variety of scenarios, e.g. multi-language projects or application selection process among candidates in different languages. We focus on the challenges related to comparing the maintainability quality typically through a maintainability index or technical debt approaches. We identify and discuss how to manage a number of challenges to produce comparable maintainability assessments across languages related to the programming paradigm (purely procedural vs OO vs multi-paradigm), the coverage of key quality dimensions, and the use of generic metrics vs more languages specific rules. Our work is based on a set of code analysis carried out in Wallonia over the past 15 years.