PatchTrack: A Comprehensive Analysis of ChatGPT's Influence on Pull Request Outcomes

May 12, 2025 Β· Declared Dead Β· πŸ› arXiv.org

πŸ‘» CAUSE OF DEATH: Ghosted
No code link whatsoever

"No code URL or promise found in abstract"

Evidence collected by the PWNC Scanner

Authors Daniel Ogenrwot, John Businge arXiv ID 2505.07700 Category cs.SE: Software Engineering Citations 4 Venue arXiv.org Last Checked 4 months ago
Abstract
The rapid adoption of large language models (LLMs) like ChatGPT has introduced new dynamics in software development, particularly within pull request workflows. While prior research has examined the quality of AI-generated code, little is known about how developers actually use these suggestions in real-world collaboration. We analyze 338 pull requests from 255 GitHub repositories containing self-admitted ChatGPT usage, including 645 AI-generated snippets and 3,486 developer-authored patches. We introduce PatchTrack, a tool that classifies whether ChatGPT patches were applied, not applied, or not suggested, enabling fine-grained analysis of AI-assisted decisions. Full adoption of ChatGPT code is rare: the median integration rate was 25%. A qualitative analysis of 89 pull requests with integrated patches revealed recurring patterns of structural integration, selective extraction, and iterative refinement, showing that developers typically treat ChatGPT's output as a starting point rather than a final implementation. Even when code was not directly adopted, ChatGPT influenced workflows through conceptual guidance, documentation, and debugging strategies. Integration decisions were shaped by scope, architectural fit, contributor role, and review norms. This study offers empirical insight into how generative AI is used in collaborative software development, showing that its impact extends beyond patch generation to broader decision-making. Our findings inform the design of AI-assisted tools, clarify patch adoption behavior, and support more transparent and effective use of LLMs in practice.
Community shame:
Not yet rated
Community Contributions

Found the code? Know the venue? Think something is wrong? Let us know!

πŸ“œ Similar Papers

In the same crypt β€” Software Engineering

Died the same way β€” πŸ‘» Ghosted