Automated Vulnerability Injection in Solidity Smart Contracts: A Mutation-Based Approach for Benchmark Development

The security of smart contracts is critical in blockchain systems, where even minor vulnerabilities can lead to substantial financial losses. Researchers proposed several vulnerability detection tools evaluated using existing benchmarks. However, most benchmarks are outdated and focus on a narrow set of vulnerabilities. This work evaluates whether mutation seeding can effectively inject vulnerabilities into Solidity-based smart contracts and whether state-of-the-art static analysis tools can detect the injected flaws. We aim to automatically inject vulnerabilities into smart contracts to generate large and wide benchmarks. We propose MuSe, a tool to generate vulnerable smart contracts by leveraging pattern-based mutation operators to inject six vulnerability types into real-world smart contracts. We analyzed these vulnerable smart contracts using Slither, a static analysis tool, to determine its capacity to identify them and assess their validity. The results show that each vulnerability has a different injection rate. Not all smart contracts can exhibit some vulnerabilities because they lack the prerequisites for injection. Furthermore, static analysis tools fail to detect all vulnerabilities injected using pattern-based mutations, underscoring the need for enhancements in static analyzers and demonstrating that benchmarks generated by mutation seeding tools can improve the evaluation of detection tools.