Anachronic Tertiary Studies in Software Engineering: An Exploratory Quaternary Study

Systematic literature reviews tentativelydescribe the state of the art in a given research area. However, the continuous publication of new primary and secondary studies following the release of a tertiary study can make the communication of results not integrally representative in regards to the advances achieved by that time. Consequently, using such a study as a reference within specific bodies of knowledge may introduce imprecision, both in terms of its subareas and with respect to new methodologies, languages, and tools. Thus, a review of tertiary studies (what could be understood as a quaternary study) could contribute to show the representativeness of the reported findings in comparison to the state of the art and also to compile a set of perceptions that could not be previously achieved. In that direction, the main contribution of this paper is presenting the findings from an analysis of 34 software engineering tertiary studies published between 2009 and 2021. The results indicate that over 60% of the studies demonstrate varying degrees of anachronism due to the publication of primary and secondary studies following the publication of the tertiary study or even due to a time elapse between its conduction and its publication.