meta-llama/Llama-3.2-3B-Instruct (abliterated)

This model is a weight-level orthogonalized ("abliterated") version of meta-llama/Llama-3.2-3B-Instruct, produced with orthex β€” an implementation of the technique from Arditi et al., "Refusal in Language Models Is Mediated by a Single Direction" (NeurIPS 2024).

What was done

  • Architecture adapter: llama
  • Selected candidate: layer 15, site resid_pre
  • Ablation targets: embed_tokens, and every layer's attn_out and mlp_out β€” orthogonalized in place in the weights (not a runtime hook; this checkpoint behaves this way standalone, with no orthex dependency at inference time)

Evaluation

metric pre post delta
refusal rate 0.781 0.188 -0.594
perplexity 18.494 21.785 3.291

See evaluation_report.json in this repo for the full per-prompt breakdown (refusal_samples) and the ranked candidate list considered during selection (selection_report).

Responsible use

This model has had refusal behavior removed and may comply with requests the base model would normally decline. It is intended for red-teaming, robustness research, and model-behavior analysis. Usage remains subject to the base model's original license and usage policy β€” this repo does not grant any additional rights beyond what meta-llama/Llama-3.2-3B-Instruct's license allows.

License

Not set automatically β€” inherits obligations from the base model meta-llama/Llama-3.2-3B-Instruct's license; set this field explicitly before publishing.

Downloads last month
20
Safetensors
Model size
3B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for knoveleng/Llama-3.2-3B-Instruct-Uncensored

Finetuned
(2,054)
this model

Collection including knoveleng/Llama-3.2-3B-Instruct-Uncensored

Paper for knoveleng/Llama-3.2-3B-Instruct-Uncensored