I do not like these "surgical removals" and would rather prefer a pass over from a tool like Heretic. These surgical removals often trigger and analyze the activated neurons and erase them. This worked fine on older models where a single refusal vector existed. Now these "abliterated" models all suffer from catastrophic breakage because they are not as simple anymore. HauhauCS (on HF) for example, makes great uncensored models although they often work on smaller models rather than large ones like this.
Heretic is a tool that removes censorship (aka "safety alignment") from transformer-based language models without expensive post-training. It combines an advanced implementation of directional ablation, also known as "abliteration", with a TPE-based parameter optimizer powered by Optuna.
This approach enables Heretic to work completely automatically. Heretic finds high-quality abliteration parameters by co-minimizing the number of refusals and the KL divergence from the original model. This results in a decensored model that retains as much of the original model's intelligence as possible. Using Heretic does not require an understanding of transformer internals. In fact, anyone who knows how to run a command-line program can use Heretic to decensor language models.
Abliteration seems to be what Heretic does?
Now these "abliterated" models all suffer from catastrophic breakage because they are not as simple anymore.
I'm not knowledgeable about each step in the process of making these abliterated models, but some more popular ones with steps after Heretic, seem to improve on the benchmarks tried of the base model:
I'm not seeing any similar benchmarks of the HauhauCS models, at least the ones I checked, so I assumed the opinion is based on your own trials, but then you argue in favour Heretic. Is the based on pre-Heretic abliteration techniques? Which might then not be appliable to this "Proprietary weight-level abliteration developed by the dealignai research team."?
Heretic is an awesome tool and I'd prefer Heretic over the "Proprietary weight-level abliteration developed by the dealignai research team" or manual labor model surgery. Since Heretic models are not often called "Abiterated" on HF but are called "Uncensored" or "Heretic" I may have gotten a bit confused there. From what I know, HauhauCS also uses Heretic so it should be the same as Heretic ones in benchmarks.
Comments
I do not like these "surgical removals" and would rather prefer a pass over from a tool like Heretic. These surgical removals often trigger and analyze the activated neurons and erase them. This worked fine on older models where a single refusal vector existed. Now these "abliterated" models all suffer from catastrophic breakage because they are not as simple anymore. HauhauCS (on HF) for example, makes great uncensored models although they often work on smaller models rather than large ones like this.
Abliteration seems to be what Heretic does?
I'm not knowledgeable about each step in the process of making these abliterated models, but some more popular ones with steps after Heretic, seem to improve on the benchmarks tried of the base model:
https://huggingface.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-...I'm not seeing any similar benchmarks of the HauhauCS models, at least the ones I checked, so I assumed the opinion is based on your own trials, but then you argue in favour Heretic. Is the based on pre-Heretic abliteration techniques? Which might then not be appliable to this "Proprietary weight-level abliteration developed by the dealignai research team."?
Heretic is an awesome tool and I'd prefer Heretic over the "Proprietary weight-level abliteration developed by the dealignai research team" or manual labor model surgery. Since Heretic models are not often called "Abiterated" on HF but are called "Uncensored" or "Heretic" I may have gotten a bit confused there. From what I know, HauhauCS also uses Heretic so it should be the same as Heretic ones in benchmarks.
And you think heretic is not abliteraterating models?
No, what I mean is i trust Heretic's abliteration more than a human's.