Robust detection of watermarks for large language models under human edits

原文摘要

Abstract Watermarking is an effective approach to distinguishing text generated by large language models (LLMs) from human-written text. However, the pervasive presence of human edits on LLM-generated text dilutes watermark signals, thereby significantly degrading detection performance of existing methods. In this paper, by modelling human edits through mixture model detection, we introduce a new method—a truncated goodness-of-fit test (Tr-GoF) for detecting watermarked text under human edits. We prove that Tr-GoF achieves optimality in robust detection of the Gumbel-max watermark in a certain asymptotic regime of substantial text modifications and vanishing watermark signals. Importantly, Tr-GoF achieves this optimality adaptively without requiring precise knowledge of human edit levels or probabilistic specifications of LLMs, unlike the optimal but impractical Neyman–Pearson likelihood ratio test. Moreover, we establish that Tr-GoF attains the highest detection efficiency rate under moderate text modifications. In contrast, sum-based detection rules used by existing methods fail to achieve optimal robustness in both regimes because the additive nature of their statistics is less resilient to edit-induced noise. We demonstrate Tr-GoF’s competitive and sometimes superior performance on synthetic data and open-source LLMs in the OPT and LLaMA families.

摘要来源：Crossref