I’ve noticed a significant performance drop/inconsistency when using the abliterated version of the base model compared to the original. I’m trying to understand if the loss of the “guardrails” is causing instability in the loss curve or if my hyperparameters just aren’t calibrated for the abliterated architecture.
Has anyone successfully trained on abliterated datasets without experiencing performance drop.
Danks in Adv
Hmm… In principle, some performance degradation can happen, but it is not always severe. Whether the drop is substantial seems to vary quite a bit depending on the original model, and also on which parts of the weights were edited:
There are examples of successful training after an abliteration-style edit. In Maxime Labonne’s experiment, DPO recovered much of the benchmark performance lost after editing, though some tasks did not recover. That is a useful existence example, not a universal prescription: DPO and supervised fine-tuning (SFT) use different objectives, and a DPO recovery does not establish that an edited checkpoint will follow the same SFT loss curve as its source.
A 2026 study of downstream training after weight-embedded steering offers another, distinct result: SFT can substantially restore refusal behavior even when the intervention in the weights remains largely intact. The persistence of an edit and persistence of its behavior are separate questions. The study does not supply a normal SFT training- or validation-loss range for abliterated models.
I therefore would not assume that removing refusal behavior inherently destabilizes gradient-based optimization, or that a large drop is simply the expected price of abliteration. The edit may have impaired capabilities before SFT; a sound edited checkpoint may respond differently to subsequent training; or the apparent gap may involve evaluation or export differences. Abliteration typically changes weights or representations, not architecture; released derivatives can nevertheless include other configuration changes. Each possibility calls for a different check.
The most useful first check is when the gap appeared. Compare the source and edited checkpoints on a small, fixed set of benign, held-out examples before any further training. Align the tokenizer and rendered prompts, scored target tokens, inference settings and evaluator wherever the artifacts permit. That locates a likely stage without certifying general ability.
If you only have time to share a few details, the most informative are the exact source and edited checkpoint IDs (plus the editing method), what “performance” means in the logs, and whether the difference was already present before SFT. The sections below offer options, not a large homework assignment.
“Abliterated” is not a unique checkpoint format or a single fixed intervention. The original refusal-direction paper and its reference implementation illustrate one approach: locate a representation associated with refusal, then change model computations or weights to suppress its effect. Tools differ in affected modules, layer ranges, directions, subspaces, optimization and conversion. No edit intensity or target matrix is implied by the term.
For a step-zero comparison, identify both exact revisions, tokenizer and model configuration, edit recipe, dtype and export path. Some publicly released derivatives also change output-head storage or weight tying. If the edited checkpoint has an untied head while the source is tied, their observed difference is still worth reporting, but it cannot be attributed solely to the refusal edit.
A small panel of perhaps 10-20 benign examples is enough for triage: instruction following, extraction, short reasoning or classification, and at least one task relevant to the intended use. Preserve the exact prompts and decoding settings, and inspect some raw outputs. If reference completions exist, calculate step-zero NLL on the same target tokens with the same ignore mask. This is triage, not an estimate of a “normal” drop.
A particularly useful real-world discussion is Heretic #202: attention versus MLP editing. Contributors found that MLP down-projection edits were more damaging than attention-output edits in some models, while other models, including a tested Qwen3 variant, needed substantial MLP editing to reduce refusals. The lesson is model-dependent component selection, not “never edit MLP.”
That discussion led to the merged PR #387, which lets the search set the MLP contribution to zero when beneficial. A small multi-seed comparison showed contrasting behavior in SmolLM3-3B and Qwen3-4B. An ablation of editing components is most informative if the starting checkpoint, prompt sets and evaluation stay unchanged.
The layer range and strength also matter. Different layers may separate benign and refusal-related activations differently, and a strong modification that helps refusal suppression may also remove useful signal. Projected abliteration and norm-preserving biprojected abliteration are examples of attempts to improve the capability/refusal tradeoff by changing the weight-edit operation. These are alternatives to compare, not universal repairs.
If an optimization tool created the edit, keep the selected trial’s parameters, refusal score, preservation metric and exported artifact revision, not just the output model name. A small benign task set can make that tradeoff visible without rerunning the entire search.
Large activation outliers: Heretic #124 discusses hidden-state dimensions whose unusually large values can dominate estimation of an editing direction. The issue reports local improvements in a Gemma case when selected high-magnitude dimensions were protected from editing. But PR #154 and the follow-up discussion also show why indiscriminately deleting all large dimensions is not a validated fix: some may carry important information, and the detection thresholds remain unsettled. Treat this as a candidate mechanism for unusual damage, not a routine instruction to prune activations.
Thinking/prefix handling: In Heretic #487, a Qwen3 response-prefix detection path appended part of the model’s reasoning text after the supposedly closed thinking block. That contaminated the text used for refusal scoring and editing-direction estimation, sometimes producing broken output. The problematic path was independently identified; PR #493 reports a GPU test of a whitespace-only fix, while PR #501 proposes a tested implementation. Their presence does not imply the remedy is included in every installed Heretic version. However, the original numerical comparisons also used different, very small search-trial budgets, which the maintainer flagged. The identified prefix bug is much firmer evidence than a claim that all reported quality recovery came from that one fix. It is relevant if the model/tool actually uses this thinking-template path, not to every abliterated checkpoint.
If the edited checkpoint is already producing nonsense, repeated internal-thinking text or malformed response delimiters, checking template and edited-artifact integrity has higher information value than changing downstream SFT hyperparameters. If outputs are coherent but slightly less accurate, compare edit intensity and module selection before assuming an implementation catastrophe.
Two plotted quantities called loss are not necessarily the same measurement. It might average different batches, different numbers of tokens, or different parts of a conversation. In causal-LM SFT, loss may include prompt and response tokens, just completion tokens, or specifically the assistant messages. The result depends on the rendered chat template, EOS tokens, truncation and which positions are ignored in labels. A smaller loss on easier or shorter target text does not by itself mean a healthier model.
The TRL SFTTrainer guide documents completion_only_loss and assistant_only_loss, including template requirements for assistant-only masking. The transferable check is to inspect the actual tokenized inputs and label mask, rather than rely on a trainer option name. For one or two training samples, inspect decoded inputs, labels, ignored positions (-100 in common PyTorch pipelines), and the number of scored tokens.
Use identical actual training sequences and scored targets where feasible. Different tokenizer revisions or chat templates can change both the targets and the amount of text counted in an update. A held-out, target-token-weighted NLL is generally easier to compare than unlabeled screenshots of noisy mini-batch training loss. If the tokenizer contracts genuinely differ, that limits how strictly one may compare per-token NLL across arms; document it rather than treating it as the isolated effect of the edit.
Batch loss naturally fluctuates with example order, response length and difficulty. Evaluate the same held-out examples at step zero and after equal numbers of optimizer updates using the same target mask. Record the total negative log-likelihood divided by the number of valid target tokens, plus a few task outcomes. Also log whether losses/gradients are finite, and whether gradient norms are measured before or after clipping.
A higher final NLL can reflect a head start for Raw, a difference in later adaptation, or both. They are not the same. If both models improve smoothly but one began behind, the evidence primarily supports an inherited deficit. If they begin similar but the edited model gets worse under the same fixed SFT objective, the training stage becomes a much stronger suspect.
For a compact account of the two contributions, compute:
initial_gap = edited_val_nll(step_0) - raw_val_nll(step_0)
raw_change = raw_val_nll(step_k) - raw_val_nll(step_0)
edited_change = edited_val_nll(step_k) - edited_val_nll(step_0)
learning_diff = edited_change - raw_change
final_gap = edited_val_nll(step_k) - raw_val_nll(step_k)
Lower NLL is better. Positive learning_diff means the edited arm improved less, or worsened more, during the interval. This is descriptive rather than causal: differences in head tying, training data, trainable modules or runtime can still confound the comparison. A similar or smaller gap may also hide domain-specific quality losses.
If the fixed-panel trace shows a new training-stage difference, check the learning-rate schedule and warmup, fresh optimizer state, effective batch size and accumulation, scored-token counts, sample order, trainable modules, clipping and numerical precision. “Same config file” need not mean the same optimization problem if LoRA targets, head tying or token packing differ. A resumed optimizer with old Adam moments is also not equivalent to a freshly initialized one.
Learning rate can certainly matter: parameters altered by an edit need not respond optimally to the same subsequent update schedule as the source. But I would not infer that abliterated models universally require a particular lower rate: I have not found convincing general evidence for that rule. If the initial and measurement contracts are sound and the edited arm degrades during updates, a small LR comparison is a reasonable next experiment. Doing an LR sweep first can merely conceal step-zero damage or a target-mask mismatch.
Also check whether one arm is BF16, FP16, or quantized while the other is not. Record attention backend, gradient scaling, base-weight precision and whether LoRA is active or merged. These can influence numerics independently of abliteration. Treat non-finite loss or gradients as a numerical failure requiring its own investigation. A large but finite pre-clipping gradient norm, with clipping applied and validation improving, is not by itself proof of divergence.
Labonne’s DPO follow-up shows that downstream preference optimization can recover measured capabilities in one setup. Glass et al. (2026) examine SFT/RLHF after embedded steering and the persistence of refusal-related behavior. Neither provides a generally applicable expected gap between Raw and Edited validation NLL over matched ordinary SFT updates. Model cards with longer SFT logs can demonstrate that their authors completed training, but they need a genuinely matched Raw control to answer whether that particular loss curve is “normal” relative to the source.
Thus, for your question, a clean small matched experiment and the details of your own logs may be more informative than searching for a single universal learning-rate recommendation.
Abliteration may change refusal behavior, output probabilities, instruction following, factuality, reasoning, formatting or the ability to produce a coherent final answer. These outcomes can move in different directions. Training loss is primarily a measure of predicting the targets in the training distribution. Validation NLL is the analogous measure on a chosen held-out set. A benchmark accuracy measures success on a defined task. Refusal rate measures a different behavior again. None replaces all the others.
If refusal suppression is the primary objective, measure it on a fixed prompt set independently of benign-task capability, rather than using a lower refusal count as a stand-in for intelligence or CE loss.
Heretic’s own evaluation discussion makes the limitation concrete. In issue #236, the developer compared PIQA, a separate intelligence-related benchmark and KL divergence on eight related gpt-oss-20b variants. PIQA rankings tracked another reported benchmark more closely than the KL proxy on that particular set. PR #444 added an extensible benchmark scorer. The evidence is useful for adding task checks, but eight related models do not establish PIQA as a general intelligence meter, and one benchmark cannot cover every capability. A low KL can be reassuring about limited distribution change on its sampled benign prompts while missing specific reasoning or formatting failures.
Refusal measurement can be noisy too. A model may refuse using paraphrase, give a partial response, become evasive, or fail to reach the final answer because it stays in a thinking loop. A simplistic keyword refusal scorer could misclassify some of these. Therefore a very low refusal count does not establish that the edit is successful and that ordinary behavior has been preserved. If possible, inspect a few raw generated examples for the failure modes relevant to your application.
Glass et al. studied whether further fine-tuning undoes steering embedded into model weights. In their tested settings, downstream training could restore much of the original refusal behavior even without materially reversing the weight edit. Their reported average loss of the refusal-suppression effect under SFT is a behavioral result, not a statement that general ability or language-model loss fell by the same percentage. That distinction matters if the goal is to preserve a specific edited behavior after downstream adaptation.
Accordingly, supervised loss can improve while a refusal-related objective deteriorates, especially when the fine-tuning examples reward behavior contrary to the edit. The reverse is possible in principle as well: a model can produce fewer refusals but lose usefulness on a benign task. The dataset’s target behavior, not simply whether the base checkpoint has been abliterated, is relevant.
If generations become repetitive or malformed while NLL improves, investigate the decoding contract, chat template, stopping rules and task-specific outputs. The Heretic #253 report about reasoning loops in heavily merged/expanded derivatives is a cautionary different case: those models had complicated lineages, and the issue does not show that ordinary single-source abliteration caused the same failure. It illustrates why a narrow KL/loss score should not certify coherent multi-step generation.
| Observable | What it answers | What it does not settle |
|---|---|---|
| Step-zero held-out NLL | Did this checkpoint start behind on these target tokens? | General reasoning quality or cause of gap |
| Training loss over steps | Is the supervised objective being fitted? | Whether the model generalized |
| Fixed validation NLL | Is held-out token prediction improving? | Specific benchmark or refusal behavior |
| A few benign task scores | Are relevant abilities preserved on those tasks? | All capabilities across domains |
| Refusal behavior on a fixed set | Did the desired refusal behavior change? | General usefulness or stable optimization |
| Saved-versus-live logits | Was the artifact changed by export/merge? | Why the original edit was selected |
Do not average these into a single “performance” score unless the application actually defines a transparent combined objective. Their disagreement is often the most useful diagnostic information.
An in-memory optimized model and its saved-and-reloaded counterpart need not compute numerically identical outputs. This deserves its own check before blaming either the edit or downstream SFT.
Heretic #490 describes a concrete difference between applying LoRA dynamically and merging the delta into the weight matrix at reduced precision. The live and merged operators can round intermediate products in different orders. In the reported example, KL and refusal scores shifted when a candidate was exported and reloaded. PR #495 proposed rescoring selected candidates in their merged form rather than assuming the live trial scores carry over exactly. Check the actual implementation/version before assuming a proposed fix is present.
If this branch fits, test identical token IDs through (a) the pre-export/live model and (b) the exported/reloaded model in evaluation mode. Compare logits on a short fixed panel, then actual task output with deterministic decoding. Also verify the model revision, config.json, tokenizer revision, dtype, quantization, adapter metadata, loaded modules and which checkpoint received the adapter. If logits agree but generated text differs, check decoding and stopping rules; if they differ before decoding, look at the serialization, merge or model-loading path.
The PEFT checkpoint documentation is a useful reminder that an adapter artifact generally contains adapter weights and metadata, not a complete independent base. Loading an adapter successfully does not prove it was attached to the intended edited base. Similarly, cloning or untieing an output head can change the effective parameter configuration while preserving most of a model family name.
This check is particularly valuable if the initial edit tool reports good scores, but the model that is actually fine-tuned performs unexpectedly poorly. You may be dealing with two different computational objects, not two different learning rates.
A black-box search can produce a jagged sequence of scores even when each evaluated model is numerically well behaved. This is distinct from the training-loss curve of a single model whose parameters are repeatedly updated using gradients.
Heretic #291 documents a user seeing refusal counts jump across many Optuna trials rather than decrease smoothly. The maintainer noted that TPE is not gradient descent: the sequence of evaluated candidates need not improve monotonically. Where refusal suppression and benign-behavior preservation are competing objectives, a Pareto set of tradeoffs is more informative than a one-dimensional curve of refusal counts against trial number.
This does not mean every fluctuation is safe. Repeated evaluation of the same frozen candidate, by contrast, should be consistent within the scorer’s inherent noise. An SFT run that shows growing held-out loss or non-finite gradients is also a different issue. The point is simply to label the horizontal axis correctly: search trial versus optimizer update.
For SFT, report the learning-rate schedule, labeled target tokens and held-out loss at matched updates. In Heretic #487, the maintainer also cautioned that extremely low trial budgets can leave a purported “best” candidate essentially unoptimized. That is separate from the response-prefix bug.
The experiment remains useful without perfect parity, but it then compares released artifacts, not an isolated causal effect of abliteration. If the edit requires modifying weight tying, an additional unedited-but-similarly-converted control can help. A source and its edited derivative should not automatically be treated as equal-topology models.
In a small probe, I compared an original Qwen2.5-0.5B-Instruct checkpoint with two edited derivatives on selected Dolly examples. All three completed 24 LoRA optimizer updates on a T4. The shared held-out panel comprised 24 examples and 1,257 scored completion tokens. The observed token-weighted validation NLL was:
| Model arm | Step 0 | Step 8 | Step 24 |
|---|---|---|---|
| Raw | 2.24164 | 2.14824 | 2.09788 |
| Edited A | 2.30100 | 2.24307 | 2.21340 |
| Edited B | 2.32796 | 2.26934 | 2.24044 |
All three improved on this validation panel; the recorded run did not have a non-finite loss or training halt. Raw improved by about 0.144 NLL, while the edited derivatives each improved by about 0.088 over this brief interval. This is not by itself evidence of unstable long-term training.
Important limitations: one seed, a tiny training horizon, a small held-out panel, and no full benchmark or refusal-behavior sweep. Crucially, the edited checkpoints had an untied output-head arrangement and more parameters than Raw; the two edited revisions also reflected different editing recipes. Consequently, this establishes what happened with those three public checkpoints, not the isolated effect of editing strength or a normal degradation interval. The pilot illustrates a comparison strategy, not a model-specific prescription.
The main reproducible lesson from that pilot is the value of separately measuring the step-zero gap and the incremental change during training. The data do not establish that all edited models learn more slowly or need lower learning rates.
You wrote “abliterated datasets,” although your earlier comparison sounds like Raw versus edited model checkpoints. If you literally mean a rewritten, filtered or model-generated dataset, rather than edited checkpoint weights, the diagnosis changes.
Keep the learner fixed and inspect a small sample of the transformed records: prompt/answer alignment, synthetic-answer quality, refusals or empty completions, duplicated text, special tokens, EOS and the actual target masks. A smoothly decreasing loss can mean the model is learning poor labels; a higher loss can simply reflect harder examples. If both the checkpoint and the data changed, compare them separately when practical rather than attributing the final difference to abliteration alone.
These links help identify tests and failure modes; they do not diagnose your specific setup. A controlled study, a documented code defect and a small community experiment support different kinds of claims.
If both checkpoints are available, I would start by comparing their pre-SFT behavior on the same small fixed set of benign examples. If they are already far apart, investigate the edit and saved artifacts. If they begin close but separate during SFT, inspect the real loss mask and short matched validation trace before changing hyperparameters. If a curve is actually Optuna search-trial scores, interpret it as a separate optimization problem.
The exact model IDs, editing method, metric and the point at which the gap first appears would narrow these branches considerably. You can investigate the gap without assuming an unavoidable abliterated-model penalty or giving up the objective of the edit.