Overview of the Wikidata Vandalism Detection Task at WSDM Cup 2017

December 16, 2017 · The Cartographer · 🏛 arXiv.org

"No code URL or promise found in abstract"
"Title-pattern auto-detect: Overview of the Wikidata Vandalism Detection Task at WSDM Cup 2017"

Evidence collected by the PWNC Scanner

Authors Stefan Heindorf, Martin Potthast, Gregor Engels, Benno Stein arXiv ID 1712.05956 Category cs.IR: Information Retrieval Citations 12 Venue arXiv.org Last Checked 3 days ago

Abstract

We report on the Wikidata vandalism detection task at the WSDM Cup 2017. The task received five submissions for which this paper describes their evaluation and a comparison to state of the art baselines. Unlike previous work, we recast Wikidata vandalism detection as an online learning problem, requiring participant software to predict vandalism in near real-time. The best-performing approach achieves a ROC-AUC of 0.947 at a PR-AUC of 0.458. In particular, this task was organized as a software submission task: to maximize reproducibility as well as to foster future research and development on this task, the participants were asked to submit their working software to the TIRA experimentation platform along with the source code for open source release.