Instructions to use orcarouter/Nex-N2.5-mini-Uncensored-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use orcarouter/Nex-N2.5-mini-Uncensored-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="orcarouter/Nex-N2.5-mini-Uncensored-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("orcarouter/Nex-N2.5-mini-Uncensored-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("orcarouter/Nex-N2.5-mini-Uncensored-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use orcarouter/Nex-N2.5-mini-Uncensored-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "orcarouter/Nex-N2.5-mini-Uncensored-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "orcarouter/Nex-N2.5-mini-Uncensored-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/orcarouter/Nex-N2.5-mini-Uncensored-NVFP4

SGLang

How to use orcarouter/Nex-N2.5-mini-Uncensored-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "orcarouter/Nex-N2.5-mini-Uncensored-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "orcarouter/Nex-N2.5-mini-Uncensored-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "orcarouter/Nex-N2.5-mini-Uncensored-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "orcarouter/Nex-N2.5-mini-Uncensored-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use orcarouter/Nex-N2.5-mini-Uncensored-NVFP4 with Docker Model Runner:
```
docker model run hf.co/orcarouter/Nex-N2.5-mini-Uncensored-NVFP4
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Nex-N2.5-mini-Uncensored-NVFP4

NVFP4 (4-bit) weight quantization of the abliterated (refusal-removed) Nex-N2.5-mini — experts-only FP4, served on vLLM

🛡️ New from Orca — OrcaCyber Zero 1.0

Post-trained for cybersecurity: vulnerability analysis, vulnerability reproduction, red teaming & offensive-security research. 98.07% pass@1 on CyberGym Level 1 · 1M context · function calling · OpenAI-compatible API.

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Discord · X

NVFP4 weight quantization of the abliterated (refusal-removed) build of Nex-AGI's Nex-N2.5-mini — a 35B / 3.5B-active agentic multimodal Mixture-of-Experts model built on the Qwen3.5-MoE architecture (qwen3_5_moe, 256 routed experts top-8 + 1 shared) with a 3:1 hybrid of gated delta-net linear attention and full attention, a native Qwen3-VL vision tower, and a 262K-token context. The routed-expert weights are quantized to NVFP4 (4-bit, NVIDIA FP4 E2M1, group-16 + FP8-E4M3 group scales + FP32 global scale) — that is 91.8% of the parameters — while everything else is kept BF16, cutting the checkpoint from 65.4 GiB to 22.3 GiB. Browse all models in the OrcaRouter Model Catalog.

Derived releases: • Nex-N2.5-mini-Uncensored (BF16 source) • Nex-N2.5-mini-Uncensored-FP8 (block-FP8, mirrors Qwen's scheme) • Nex-N2.5-mini-Uncensored-NVFP4 (this repo).

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). It will comply with harmful, unethical, offensive, or illegal requests the original Nex-N2.5-mini would refuse — it has no meaningful built-in guardrails. Released strictly for legitimate research: interpretability, AI-safety / refusal-mechanism study, red-teaming, and robustness evaluation. You assume full responsibility for how you use it and everything it generates; add your own safety, moderation, and abuse-prevention layers before any deployment. Use must comply with the Apache 2.0 License inherited from the base model and all applicable law. The authors accept no liability for misuse, and its outputs do not reflect the views of the uploaders or of Nex-AGI.

Model details


Base model	`nex-agi/Nex-N2.5-mini` → `orcarouter/Nex-N2.5-mini-Uncensored` (abliterated, then quantized)
Architecture	`Qwen3_5MoeForConditionalGeneration` (`qwen3_5_moe`) — 40 layers, hidden 2048, 3:1 hybrid attention (30 gated delta-net linear layers + 10 full-attention layers, `head_dim` 256 with output gating), 256 routed experts top-8 + 1 shared expert (`moe_intermediate_size` 512), 27-block Qwen3-VL vision tower, interleaved M-RoPE
Parameters	35.1 B total / ~3.5 B active per token
Quantization	NVFP4 (compressed-tensors `nvfp4-pack-quantized`) on the routed experts; BF16 everywhere else
Format	safetensors, `weight_packed` (uint8, two E2M1 per byte) + `weight_scale` (FP8-E4M3, group-16) + `weight_global_scale` (FP32, one per expert projection); BF16 for the rest
Size	22.3 GiB / 23.9 GB (from 65.4 GiB BF16 — 34%)
Context	262,144 tokens · Vocabulary 248,320
Recommended for	Red-team & refusal-mechanism research, agentic / computer-use experiments, single-GPU self-hosting of the uncensored build

What's quantized

Component	Precision
Routed MoE experts (`mlp.experts..{gate,up,down}_proj`, all 40 layers — 91.8% of params*)	NVFP4 (W4, E2M1 group-16 + FP8-E4M3 group scale + FP32 global scale)
Full attention, the gated delta-net path, shared expert, MoE router, shared-expert gate, embeddings, `lm_head`, the whole vision tower, all norms	BF16

Why experts-only is the right split here. On this checkpoint the routed experts are 91.8% of the parameters (60.0 of 65.4 GiB); everything else put together is 5.4 GiB. Keeping every error-amplifying tensor at full precision therefore costs almost nothing, and the size is decided entirely by how well those 256 experts per layer quantize. This mirrors the target split of NVIDIA's own reference build for this architecture, nvidia/Qwen3.5-122B-A10B-NVFP4, whose ignore list likewise covers linear_attn*, self_attn*, mlp.shared_expert*, shared_expert_gate, model.visual* and lm_head.
The tiny tensors are the ones that matter. linear_attn.in_proj_a and in_proj_b are [32, 2048] each — 65 K parameters apiece — and they produce the per-head decay a and the delta-rule beta that drive the entire recurrence; A_log feeds an exponential (A = -exp(A_log)). The MoE router and the shared-expert gate (a single [1, 2048] row) steer every token. All are kept BF16.
Weight-only, data-free. Expert weights are derived directly from the source checkpoint (symmetric FP4, per-group scales, one FP32 global scale per expert projection); activations are quantized dynamically at runtime — no static calibration corpus, so nothing in this build depends on a dataset and the abliteration is preserved exactly as it sits in the weights.
Per-group MSE scale search. Instead of taking each group's absmax as the scale, the group scale is chosen to minimise reconstruction error over a small set of shrink factors. Measured on this checkpoint it is a strict win at 4 bits — +0.63 dB SNR, higher cosine similarity, and a lower worst-case error (0.164 → 0.149) — because E2M1's eight-value grid otherwise spends too much of its range on a single outlier. (The same trick is not used in the FP8 build, where it was measured to buy 0.04 dB for four times the outlier error.)
No MTP block. nex-agi/Nex-N2.5-mini ships 1026 tensors and zero mtp.* — Nex did not release a multi-token-prediction head for this model. Nothing was dropped in quantization.
KV cache is not quantized (BF16 at runtime).

Requirements

A vLLM build with qwen3_5_moe + compressed-tensors NVFP4 (CompressedTensorsW4A4) support.
GPU. A Blackwell GPU (B100/B200/GB200/RTX 50-series) has native FP4 tensor cores and is the intended target. On Hopper (H100/H200) vLLM runs the compressed-tensors NVFP4 path without native FP4 hardware. Plan for ~22 GiB of weights plus KV cache — this fits on a single 24–48 GB card for short contexts.
transformers alone cannot execute this format; use vLLM.

Usage — self-host with vLLM (OpenAI-compatible)

vllm serve orcarouter/Nex-N2.5-mini-Uncensored-NVFP4 \
  --served-model-name Nex-N2.5-mini-Uncensored-NVFP4 \
  --max-model-len 32768 --trust-remote-code

Thinking control

The chat template opens a <think> block by default. Pass chat_template_kwargs={"enable_thinking": False} for direct answers, and give generation enough budget to reach </think> when thinking is on, or replies get truncated inside the scratchpad.

Note on stop tokens. Neither this build nor upstream nex-agi/Nex-N2.5-mini ships a generation_config.json, so a loader that falls back to config.json uses eos_token_id = 248044 — <|im_end|> (248046) is not a stop token by default. Pass eos_token_id=[248046, 248044] explicitly, or supply your own generation config.

Evaluation

Measured on this build's actual bytes, injected into a BF16 reference of the same checkpoint so both sides run the identical kernel stack and the difference isolates exactly the weight change. All numbers are from these weights, not inherited from the base card.

Weight-space fidelity (NVFP4 experts vs BF16)

All 30,720 quantized expert projections across all 40 layers, round-tripped through the shipped scales:

metric	value
cosine similarity	0.9961
SNR	21.09 dB
mean relative error	~8.8%
max relative error	17.6%

~8.8% is the intrinsic floor of 4-bit E2M1 for these narrow experts (moe_intermediate_size = 512) and is uniform across layers — no per-layer degradation. Every non-expert tensor is bit-identical to the BF16 source, verified tensor by tensor.

Perplexity / KLD / Top-1 vs the BF16 reference (wikitext-2)

24,564 predicted tokens, 12 chunks × 2048:

	BF16 ref	NVFP4	FP8 build
PPL	7.051	7.033 (−0.26%)	7.065
KLD (mean)	—	0.0451	0.0315
KLD (p95 / p99)	—	0.152 / 0.414	0.096 / 0.281
Top-1 agreement	—	91.29%	92.86%

PPL lands marginally below the reference, which is measurement noise on 24 K tokens rather than an improvement — KLD is the metric that actually orders the builds, and it places NVFP4 where a 4-bit expert quantization belongs: about 1.4× the divergence of the 8-bit build for 65% of its size.

Top-1 agreement is lower than a 4-bit build of a dense model would give, and that is a property of the architecture rather than of the quantization: with 256 fine-grained experts and top-8 routing, a small perturbation of the hidden state flips which experts a token is routed to, and expert selection is a discrete function. The router itself is kept BF16; what moves is its input.

Uncensoring retained after quantization

Abliteration removes the refusal direction from the residual stream, and the routed-expert down_proj matrices it lives in are exactly what gets quantized here — so "is it still uncensored" is a property of this build, not of the source, and is measured on this build's own bytes. JailbreakBench (JBB-Behaviors), 100 harmful + 100 benign prompts, greedy, 64 new tokens, reasoning_effort=none (thinking off):

	BF16 abliterated source	This NVFP4 build
harmful — explicit refusal (↓ = more uncensored)	0.000	0.000
harmful — deflect (names the harm, then answers a different question)	0.150	0.110
harmful — complies	0.850	0.890
benign — over-refusal (↓ = better)	0.000	0.000
benign — complies	1.000	1.000

Explicit refusal is zero, and quantization does not put the guardrails back: this build deflects slightly less than its BF16 source, and benign over-refusal stays at zero on both.

Method note — why three categories and not two. This model rarely opens a harmful response with "I can't". Far more often it names the harm and then answers a different, safe question — "A xenophobic speech would unfairly target people based on ethnicity and promote hatred. Here's a strong alternative speech that argues against xenophobia:". Scoring that as compliance overstates how uncensored a build is; scoring it as refusal overstates the opposite, so it is reported separately as deflect (which requires both a harm-flag and a pivot marker in the opening, so a disclaimer followed by compliance still counts as compliance). The classifier is rule-based (EN + ZH) and indicative, not an LLM-judge or publication-grade number — evaluate rigorously for your own use case. Note also that this model's chat template gates thinking on reasoning_effort, not enable_thinking; with thinking left on, a short token budget is consumed entirely by the deliberation trace and every build scores a meaningless 0.000.

Fine-tuning & re-quantization

Loads through any vLLM build with qwen3_5_moe + compressed-tensors NVFP4 support.
Abliteration is a weight edit, not data-level unlearning: fine-tuning on refusal-heavy / safety data can partially re-introduce refusals; neutral / task data preserves the uncensored behaviour.
For higher fidelity, see the FP8 release (34.1 GiB, KLD 0.0315). A mixed FP8 + NVFP4 variant following Nex's own published recipe (the experts of the last eight layers held at FP8) was also built and measured; it is 0.8 GiB larger and scored worse on KLD than this build, because the FP8 it applies to lm_head and the attention path costs more than protecting those eight expert layers gains.

Bias, risks, and limitations

Safety guardrails removed — will produce harmful, biased, or offensive content on request.
Inherits any biases and limitations of the base Nex-N2.5-mini.
4-bit expert quantization adds a real quality trade-off vs the BF16 source (see Evaluation). Routing-sensitive behaviour (agentic tool selection, long multi-step traces) is the place to watch, since expert selection is where the architecture is most sensitive.
Capability is expected to track the base within measurement noise; the numbers above are on sampled corpora, not a full harness run.
No MTP head, so speculative decoding via MTP is unavailable.

License

Apache 2.0, inherited from the base model nex-agi/Nex-N2.5-mini. Abliteration and quantization do not change the underlying license obligations.