Adversarial Semantic Collisions

November 09, 2020 · Entered Twilight · 🏛 Conference on Empirical Methods in Natural Language Processing

"Last commit was 5.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: README.md, assets, collision_ext_sum.py, collision_paraphrase.py, collision_polyencoder.py, collision_retrieval.py, constant.py, models, requirements.txt, scripts, utils

Authors Congzheng Song, Alexander M. Rush, Vitaly Shmatikov arXiv ID 2011.04743 Category cs.CL: Computation & Language Cross-listed cs.CR Citations 58 Venue Conference on Empirical Methods in Natural Language Processing Repository https://github.com/csong27/collision-bert ⭐ 25 Last Checked 2 months ago

Abstract

We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models. We develop gradient-based approaches for generating semantic collisions and demonstrate that state-of-the-art models for many tasks which rely on analyzing the meaning and similarity of texts-- including paraphrase identification, document retrieval, response suggestion, and extractive summarization-- are vulnerable to semantic collisions. For example, given a target query, inserting a crafted collision into an irrelevant document can shift its retrieval rank from 1000 to top 3. We show how to generate semantic collisions that evade perplexity-based filtering and discuss other potential mitigations. Our code is available at https://github.com/csong27/collision-bert.