# How to Implement Text Watermarks for LLM-Generated Content: A Complete Guide > Learn how to implement text watermarks for LLM generated content with the Lordog/dive-into-llms repository. Discover a modular pipeline for embedding, detecting, and evaluating watermarks using KGW, X-SIR, and SIR algorithms. - Repository: [Tongxin Yuan/dive-into-llms](https://github.com/Lordog/dive-into-llms) - Tags: how-to-guide - Published: 2026-04-16 --- **The dive-into-llms repository provides a modular, three-stage pipeline—embedding, detection, and evaluation—to implement text watermarks for LLM-generated content using statistical algorithms like KGW, X-SIR, and SIR.** Text watermarks for LLM-generated content embed invisible statistical signatures during token generation, enabling traceability without degrading text quality. The `Lordog/dive-into-llms` repository provides a production-ready implementation in `documents/chapter5/` that supports multiple watermark algorithms and includes robustness testing against paraphrasing and translation attacks. This guide walks through the complete technical workflow using the official command-line tools and Jupyter notebook. ## Overview of the Watermark Pipeline The implementation organizes text watermarking into three independent stages that can be integrated into larger training or inference systems. ### Stage 1: Watermark Embedding The embedding stage modifies the model's token distribution during generation to insert statistical patterns. The [`watermarker.py`](https://github.com/Lordog/dive-into-llms/blob/main/watermarker.py) script wraps Hugging Face models and injects watermark-specific logic at each decoding step based on the `WATERMARK_METHOD_FLAG` variable. Supported algorithms include: - **KGW** (Kirchenbauer-Green-Geiping-Wen): Splits vocabulary into green/red lists based on previous tokens - **X-SIR**: Cross-token statistical watermarking - **SIR**: Sentence-level information retrieval watermarking The script outputs JSONL files containing generated samples with embedded markers saved to `gen///`. ### Stage 2: Watermark Detection Detection quantifies watermark presence using **z-scores**. The [`detect_watermark.py`](https://github.com/Lordog/dive-into-llms/blob/main/detect_watermark.py) script reads generated JSONL files and computes statistical tests to determine if text contains the embedded signature. A high positive z-score indicates strong watermark presence, while scores near zero suggest clean text. This stage operates independently from generation, allowing you to test arbitrary text strings against your watermark keys. ### Stage 3: Watermark Evaluation Evaluation compares watermarked distributions against clean (human) baselines using ROC curves, AUC calculations, and accuracy metrics. The [`evaluate_watermark.py`](https://github.com/Lordog/dive-into-llms/blob/main/evaluate_watermark.py) script takes z-score files from both watermarked and clean texts to measure detection reliability and false positive rates. ## Implementing the Watermark Workflow ### Prerequisites and Setup Ensure you have access to the chapter 5 resources in the repository: ```bash cd documents/chapter5 # Dependencies include transformers, torch, and scipy for statistical tests ``` Set environment variables for your model and algorithm choice: ```bash MODEL=baichuan-inc/Baichuan-7B MODEL_ABBR=baichuan7b WATERMARK_METHOD_FLAG="--watermark_method kgw" ``` ### Generating Watermarked Text Run the watermarker to generate text with embedded signatures: ```bash python watermarker.py \ --model $MODEL \ $WATERMARK_METHOD_FLAG \ --output_file gen/$MODEL_ABBR/kgw/mc4.en.mod.jsonl ``` This creates `mc4.en.mod.jsonl` containing watermarked samples in the `gen/baichuan7b/kgw/` directory. ### Computing Detection Z-Scores Calculate statistical scores for the generated text: ```bash python detect_watermark.py \ --detect_file gen/$MODEL_ABBR/kgw/mc4.en.mod.jsonl \ --output_file gen/$MODEL_ABBR/kgw/mc4.en.mod.z_score.jsonl ``` For baseline comparison, generate scores for clean human text: ```bash python detect_watermark.py \ --detect_file gen/$MODEL_ABBR/kgw/mc4.en.hum.jsonl \ --output_file gen/$MODEL_ABBR/kgw/mc4.en.hum.z_score.jsonl ``` ### Evaluating Detection Performance Compute ROC curves and accuracy metrics by comparing watermarked versus clean distributions: ```bash python evaluate_watermark.py \ --wm_zscore gen/$MODEL_ABBR/kgw/mc4.en.mod.z_score.jsonl \ --hm_zscore gen/$MODEL_ABBR/kgw/mc4.en.hum.z_score.jsonl \ --output_dir eval/$MODEL_ABBR/kgw ``` ## Testing Robustness Against Attacks The implementation includes utilities to test watermark survival after content manipulation. ### Paraphrase and Translation Attacks Use [`paraphrase.py`](https://github.com/Lordog/dive-into-llms/blob/main/paraphrase.py) to rewrite watermarked text via GPT-3.5-turbo, simulating an evasion attempt: ```bash python paraphrase.py \ --input_file gen/$MODEL_ABBR/kgw/mc4.en.mod.jsonl \ --output_file gen/$MODEL_ABBR/kgw/mc4.en-zh.mod.jsonl ``` Re-run detection on attacked content to measure score degradation: ```bash python detect_watermark.py \ --detect_file gen/$MODEL_ABBR/kgw/mc4.en-zh.mod.jsonl \ --output_file gen/$MODEL_ABBR/kgw/mc4.en-zh.mod.z_score.jsonl ``` The `watermark.ipynb` notebook in `documents/chapter5/` demonstrates complete robustness testing workflows, including translation attacks and comparative analysis of z-score distributions before and after rewriting. ## Summary - **Three-stage architecture**: Independent CLI tools for embedding ([`watermarker.py`](https://github.com/Lordog/dive-into-llms/blob/main/watermarker.py)), detection ([`detect_watermark.py`](https://github.com/Lordog/dive-into-llms/blob/main/detect_watermark.py)), and evaluation ([`evaluate_watermark.py`](https://github.com/Lordog/dive-into-llms/blob/main/evaluate_watermark.py)) - **Algorithm flexibility**: Native support for KGW, X-SIR, and SIR watermarking methods via `--watermark_method` flags - **Statistical detection**: Z-score-based quantification of watermark strength with ROC and AUC evaluation metrics - **Attack robustness**: Built-in testing against GPT-3.5-turbo paraphrasing and translation attacks using [`paraphrase.py`](https://github.com/Lordog/dive-into-llms/blob/main/paraphrase.py) - **Modular integration**: JSONL-based I/O enables insertion into existing LLM training and inference pipelines ## Frequently Asked Questions ### What watermark algorithms does the dive-into-llms implementation support? The repository supports **KGW** (Kirchenbauer-Green-Geiping-Wen), **X-SIR**, and **SIR** algorithms. You can select these via the `--watermark_method` flag when running [`watermarker.py`](https://github.com/Lordog/dive-into-llms/blob/main/watermarker.py), as documented in [`documents/chapter5/README.md`](https://github.com/Lordog/dive-into-llms/blob/main/documents/chapter5/README.md) and demonstrated in `watermark.ipynb`. ### How does the detection script determine if text contains a watermark? The [`detect_watermark.py`](https://github.com/Lordog/dive-into-llms/blob/main/detect_watermark.py) script computes a **z-score** that measures the statistical deviation of token distributions from expected random behavior. According to the implementation in `documents/chapter5/`, a high positive z-score indicates strong watermark presence, while scores near zero suggest the text is unmarked. ### Can this implementation test watermark robustness against rewriting attacks? Yes. The repository includes [`paraphrase.py`](https://github.com/Lordog/dive-into-llms/blob/main/paraphrase.py) to simulate attacks using GPT-3.5-turbo for paraphrasing and translation. You can re-run detection on rewritten outputs to measure how z-scores degrade, allowing you to calculate robustness metrics and ROC curves under adversarial conditions. ### Where are the main implementation files located? The core implementation resides in `documents/chapter5/`, specifically `watermark.ipynb` (full Jupyter notebook pipeline), [`watermarker.py`](https://github.com/Lordog/dive-into-llms/blob/main/watermarker.py) (generation), [`detect_watermark.py`](https://github.com/Lordog/dive-into-llms/blob/main/detect_watermark.py) (statistical testing), and [`evaluate_watermark.py`](https://github.com/Lordog/dive-into-llms/blob/main/evaluate_watermark.py) (performance metrics). The accompanying [`README.md`](https://github.com/Lordog/dive-into-llms/blob/main/README.md) provides command-line reference documentation for all supported flags and workflows.