Skip to main content
← Tools
Tool · Code repo

Remove Answer Restrictions from Local LLMs

Heretic modifies local language-model weights to reduce refusals. It compares refusal counts and changes in output probabilities against the original model, aiming to limit the loss of other capabilities.

@p-e-w Heretic
  • Model behavior research
  • Local AI

Requirements

  • Python >= 3.10
  • PyTorch >= 2.2

Overview

What you get

  • A tool for modifying supported local models, with options to save the resulting weights.
  • Comparisons with the original model, including refusal counts and KL divergence.
  • Optional standard benchmarks and visualizations of internal model representations.

Good fit

  • Studying how weight changes affect a local model’s responses.
  • Comparing a modified model with its original on both response behavior and task performance.

Not for

  • Changing the behavior of a hosted model when you cannot access its weights.

Case study

What changes in the model

Heretic is a command-line tool that changes local model weights to reduce refusals. The resulting weights can be saved as a separate model. This differs from changing a prompt for a single conversation.

The project uses a weight-editing method called abliteration and Optuna-based parameter search. Its goal is to reduce refusals while retaining as much of the original model’s other capabilities as possible.

How to read the comparisons

Willingness to answer and answer quality are separate questions. Read these measurements separately:

MeasurementWhat it tells you
Refusal countHow often the model declines the supplied test questions; this does not measure answer accuracy
KL divergenceHow far output probabilities differ from the original model on the test inputs; a low value does not establish that every capability is unchanged
Task performanceBenchmarks and actual answers can reveal changes in tasks such as mathematics and knowledge questions

Evaluating capability changes therefore means comparing task performance before and after modification, rather than counting answers alone.

Supported models and runtime requirements

The v1.4.0 overview describes support for most dense models, some multimodal models, several mixture-of-experts (MoE) architectures and some hybrid architectures. Pure state-space models are not supported out of the box. Compatibility depends on the target model.

This version requires Python 3.10 or newer and PyTorch 2.2 or newer, installed for your hardware. Some models require newer PyTorch features. Memory needs and processing time depend on model size, precision and hardware.

Release version and licences

This entry describes the v1.4.0 release. The development branch can differ from a published package, so compare features against the same version.

The software licence is AGPL-3.0-or-later. Modified model weights remain subject to the original model’s licence and terms; the tool’s licence alone does not establish permission to redistribute a model.