# heretic | Philipp Emanuel Weidmann | Knowledge Base | Instagit

Fully automatic censorship removal for language models

GitHub Stars: 8.2k

Repository: https://github.com/p-e-w/heretic

---

## Articles

### [How to Customize System Prompts and Model Behavior in Heretic's Chat Interface](/p-e-w/heretic/how-to-customize-system-prompts-and-model-behavior-in-heretics-chat-interface)

Customize Heretic's chat interface system prompts and model behavior. Learn to modify config.py, use environment variables, or CLI flags for tailored AI interactions.

- Tags: how-to-guide
- Published: 2026-02-19

### [Understanding Pareto Front Results in Heretic: How to Choose Optimal Decensoring Parameters](/p-e-w/heretic/what-are-pareto-front-results-in-heretic-and-how-to-choose-the-best-parameters)

Learn how Heretic presents Pareto front results balancing refusal reduction and KL divergence. Choose optimal decensoring parameters based on your specific trade-offs for better model control.

- Tags: deep-dive
- Published: 2026-02-19

### [How to Interpret and Use Optuna Trial Results in Heretic: A Complete Guide](/p-e-w/heretic/how-to-interpret-and-utilize-the-trial-optimization-results-from-optuna-in-heretic)

Learn to interpret and use Optuna trial results in Heretic. Understand hyperparameters, KL divergence, and refusal counts to reconstruct your model state with this complete guide.

- Tags: how-to-guide
- Published: 2026-02-19

### [How to Add Support for New Model Architectures in Heretic: A Complete Guide](/p-e-w/heretic/how-can-i-add-support-for-new-model-architectures-in-heretic)

Learn how to add support for new model architectures in Heretic by modifying three key Python functions. This guide simplifies integrating custom models into the Heretic framework.

- Tags: how-to-guide
- Published: 2026-02-19

### [How to Debug Issues with Loading Prompt Datasets in Heretic](/p-e-w/heretic/how-to-debug-issues-with-loading-prompt-datasets-in-heretic)

Solve Heretic prompt dataset loading problems. Debug `DatasetSpecification`, split, and column issues using `load_prompts`. Run loading logic interactively for faster issue resolution.

- Tags: how-to-guide
- Published: 2026-02-19

### [How to Upload Decensored Models to Hugging Face Using Heretic](/p-e-w/heretic/how-to-upload-decensored-models-to-hugging-face-using-heretic)

Upload decensored models to Hugging Face securely with Heretic. This guide shows you how to use the interactive CLI to push adapters or merged models without saving credentials.

- Tags: how-to-guide
- Published: 2026-02-19

### [How to Configure Multi-GPU and Device Mapping for Heretic: A Complete Guide](/p-e-w/heretic/how-to-configure-multi-gpu-and-device-mapping-for-heretic)

Configure multi-GPU and device mapping for Heretic using device_map and max_memory in TOML, CLI, or env variables. Optimize your performance with this complete guide.

- Tags: how-to-guide
- Published: 2026-02-19

### [Row Normalization Strategies in Heretic: A Complete Guide to LoRA Abliteration](/p-e-w/heretic/what-are-the-different-row-normalization-strategies-in-heretic-and-when-to-use-them)

Explore Heretic's three row normalization strategies NONE PRE and FULL Understand how to use them to balance computational speed and weight magnitude preservation for LoRA adapters

- Tags: deep-dive
- Published: 2026-02-19

### [How Heretic's Winsorization Handles Massive Activations in Language Models](/p-e-w/heretic/how-does-heretics-winsorization-handle-massive-activations)

Learn how Heretic's winsorization effectively handles massive activations in language models by calculating per-layer quantile thresholds and capping extreme values for numerical stability.

- Tags: deep-dive
- Published: 2026-02-19

### [Benefits of Using orthogonalize_direction in Heretic for Targeted Model Abliteration](/p-e-w/heretic/what-are-the-benefits-of-using-orthogonalize_direction-in-heretic)

Learn how orthogonalize_direction in Heretic removes harmful refusal components while preserving model helpfulness for targeted abliteration. Improve model safety today.

- Tags: deep-dive
- Published: 2026-02-19

### [Understanding direction_index and Layer Selection in Heretic's Ablation Scope](/p-e-w/heretic/understanding-direction_index-and-layer-selection-in-heretics-ablation-scope)

Master Heretic's ablation scope with precise control over direction_index and layer selection. Learn how these parameters fine-tune your model's refusal directions for optimal results.

- Tags: deep-dive
- Published: 2026-02-19

### [How to Tune Batch Size and Optimize Performance in Heretic](/p-e-w/heretic/how-to-tune-batch-size-and-optimize-performance-in-heretic)

Learn how to tune batch size and optimize performance in Heretic. automatically benchmark for optimal throughput or set a max batch size to prevent memory errors. Boost your computational efficiency.

- Tags: performance
- Published: 2026-02-19

### [How Heretic Resumes Interrupted Optimization Runs: A Deep Dive into Checkpoint Recovery](/p-e-w/heretic/how-does-heretic-handle-resuming-interrupted-optimization-runs)

Learn how Heretic resumes interrupted optimization runs. Discover its journal-storage backend, checkpoint recovery, and seamless study state reloading for continuous optimization.

- Tags: deep-dive
- Published: 2026-02-19

### [How to Extend Heretic's Refusal Detection with Custom Markers](/p-e-w/heretic/how-can-i-extend-heretics-refusal-detection-with-custom-markers)

Learn to extend Heretic's refusal detection with custom markers. Modify config.toml, set environment variables, or update Settings programmatically for enhanced control.

- Tags: how-to-guide
- Published: 2026-02-19

### [How to Configure Prompt Datasets for Heretic's Evaluation and Testing](/p-e-w/heretic/how-to-configure-prompt-datasets-for-heretics-evaluation-and-testing)

Learn how to configure prompt datasets for Heretic's evaluation and testing. Explore dataset specifications in config default toml and optimize your prompts effectively.

- Tags: how-to-guide
- Published: 2026-02-19

### [Difference Between Merging a Model and Using an Adapter with Heretic](/p-e-w/heretic/what-is-the-difference-between-merging-a-model-and-using-an-adapter-with-heretic)

Understand merging models vs using adapters in Heretic. Merge bakes LoRAs into base weights for standalone checkpoints. Adapters keep LoRAs separate for lightweight experimentation. Learn the key differences now.

- Tags: deep-dive
- Published: 2026-02-19

### [What Does a High KL Divergence Score Mean in Heretic? Interpretation Guide](/p-e-w/heretic/what-does-a-high-kl-divergence-score-mean-and-how-to-interpret-it-in-heretic)

Understand what a high KL divergence score means in Heretic. Learn how this score indicates model drift and potential degradation of language capabilities and knowledge.

- Tags: deep-dive
- Published: 2026-02-19

### [What Model Architectures Does Heretic Support? Dense LLMs and Four MoE Families Explained](/p-e-w/heretic/what-model-architectures-does-heretic-support-besides-dense-and-moe)

Explore Heretic's model architecture support, including dense transformers and four MoE families like Qwen and Phi-3.5 MoE. Learn what Heretic currently excludes.

- Tags: deep-dive
- Published: 2026-02-19

### [How to Implement Model Quantization Using bnb_4bit with Heretic: A Complete Guide](/p-e-w/heretic/how-can-i-implement-model-quantization-using-bnb-4bit-with-heretic)

Implement model quantization using bnb_4bit with Heretic. This guide shows how to configure quantization via file or the Python API for efficient model loading with Hugging Face.

- Tags: how-to-guide
- Published: 2026-02-19

### [How to Use Heretic Research Features: Residual Plotting and Geometry Analysis](/p-e-w/heretic/how-to-use-heretics-research-features-like-residual-plotting-and-geometry-analysis)

Learn to use Heretic's residual plotting and geometry analysis features to visualize refusal direction. Enable these powerful research tools by configuring settings for deeper insights.

- Tags: how-to-guide
- Published: 2026-02-19

### [How to Balance KL Divergence and Refusal Reduction in Heretic Optimization](/p-e-w/heretic/how-to-balance-kl-divergence-and-refusal-reduction-in-heretic-optimization)

Learn to balance KL divergence and refusal reduction in Heretic optimization. Discover how Heretic tackles multi-objective optimization for Pareto-optimal solutions in model fidelity and safety.

- Tags: deep-dive
- Published: 2026-02-19

### [Heretic Ablation Parameters: Understanding the Four Key Controls for Model Modification](/p-e-w/heretic/what-are-the-key-ablation-parameters-in-heretic-and-how-do-they-work)

Master Heretic ablation parameters max_weight, max_weight_position, min_weight, and min_weight_distance. Learn how to precisely modify LoRA adapters and control unwanted behaviors for better model performance.

- Tags: deep-dive
- Published: 2026-02-19

### [How Heretic Calculates Refusal Directions: A Deep Dive into the Source Code](/p-e-w/heretic/how-does-heretic-calculate-refusal-directions)

Explore how Heretic calculates refusal directions by analyzing prompt activations. Discover the source code logic for identifying and understanding harmful content signals. Learn more.

- Tags: deep-dive
- Published: 2026-02-19