⚡ Swift1.5-Qwen3.8-Flash-Next-EXL3 (4.05 bpw)

High-Performance Agentic Coding & Reasoning Model by KatterMobile (Kaden)

❤️ Please leave a LIKE / HEART on this Hugging Face repository!
If you find this model, the fine-tuning, or the EXL3 quantization useful, hitting the heart button at the top of this page makes a huge difference and helps independent developers like me stay motivated to release more frontier open-source models for everyone.

☕ A Personal Note from KatterMobile

*"Hello! I'm KatterMobile (real name Kaden). I spent dozens of hours training, fine-tuning, merging, and quantizing this model on a single NVIDIA GB10 Grace Blackwell DGX Spark workstation. I am an independent researcher and developer without corporate backing. If you find this release valuable or use it in your coding workflows, please consider supporting me on Ko-fi—literally anything helps cover compute electricity and hosting costs! Thank you so much for reading and supporting independent AI."*

☕ Support on Ko-fi: https://ko-fi.com/kattermobile
📬 Support & Inquiries: support@kattermobile.com

🎬 Live Demonstration: Pi Code CLI (`pi`)

Here is Swift1.5-Qwen3.8-Flash-Next-EXL3 running live and generating clean, typed Python code in real-time inside Pi Code CLI (pi), connected directly to our local TabbyAPI instance:

⚙️ Mandatory Thinking & Sampling Parameters

Preserve Reasoning Traces: To obtain full reasoning performance and prevent premature cutoff, ensure your client or UI preserves <think> ... </think> tokens and exposes them in reasoning_content.

Parameter	Recommended Value	Purpose & Notes
`temperature`	`1.0`	Mandatory for Qwen3.8 / Swift reasoning exploration dynamics.
`top_p`	`0.95`	Nucleus sampling threshold.
`top_k`	`20`	Stabilizes token choices during multi-step reasoning.
`min_p`	`0.0`	Disabled (avoids clipping low-probability reasoning branches).
`presence_penalty`	`0.0`	Neutral presence penalty.
`repetition_penalty`	`1.0`	Disabled (prevents damaging code indentation and syntax structures).
`max_tokens`	`524,288` (512k)	Huge output budget for full repository solutions and exhaustive reasoning.
`context_window`	`1,048,576` (1M)	1 Million token context window fully supported with native YaRN RoPE.
`preserve_reasoning`	`true`	Keep internal `<think>` traces intact.

🧠 Transparent Architecture: Exactly How I Trained It

This model is not just a standard quant—it is a specialized, post-trained coding agent. Here is the exact methodology:

[ukisai/Swift1.5 Base]
         │
         ▼
[QLoRA SFT on ukisai/Qwen3.8-27B-multi-turn-agent-sft] (r=64, α=128, all linear projections)
         │
         ▼
[In-Place FP16 LoRA Fusion] (420 weight tensors permanently merged, sidecars deleted)
         │
         ▼
[EXL3 Quantization (4.05 bpw)] (Trellis & Hessian calibration for DGX Spark GB10)
         │
         ▼
[Deployment on TabbyAPI :8899] (1M Context, 512k Output, Solo MTP Drafting @ ~50–61 tok/s)

1. Distilled Reasoning Foundation

Base Model: ukisai/Swift1.5-Qwen3.8-Flash-Next, an efficient reasoning evolution of Alibaba's Qwen/Qwen3.8-Flash-Next. Swift cuts overthinking tokens by 63.4% while preserving frontier coding accuracy.

2. Multi-Turn Agentic Fine-Tuning (QLoRA)

Dataset: Trained on ukisai/Qwen3.8-27B-multi-turn-agent-sft, consisting of complex multi-turn programming dialogues, autonomous tool use, terminal interactions, shell command debugging, and iterative code correction.
Hyperparameters & Recipe:
- Framework: Unsloth QLoRA with rank $r=64$ and alpha $\alpha=128$ (dropout: 0.0).
- Target Matrices: All linear projection matrices: q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_a, in_proj_b, in_proj_z, out_proj, gate_proj, up_proj, down_proj, and shared_expert_gate.
- Router Invariance: MoE routing gates remained strictly frozen to eliminate router drift or expert leakage.
- Convergence: Training loss smoothly converged down to 0.2359 with zero gradient spikes.

3. In-Place FP16 Weight Fusion

All 420 trained LoRA delta tensors were calculated and fused directly into the base FP16 weights.
Unmerged sidecars were completely removed, eliminating adapter loading latency and guaranteeing zero runtime discrepancy.

4. Calibrated EXL3 Quantization (4.05 bpw)

Converted with Turboderp's ExLlamaV3 (EXL3) framework using second-order Hessian optimizations specifically tailored for the NVIDIA GB10 Grace Blackwell DGX Spark:

Full Self-Attention (Q/K/V/O): 6.00 bpw (ensures rock-solid attention fidelity across 1M context)
512 Routed MoE Experts: 4.00 bpw trellis quantization across all 48 transformer layers
LM Head: 6.01 bpw (preserves vocabulary projection accuracy for complex code syntax)
MTP Auxiliary Draft Head: 4.02 bpw across 512 draft experts, enabling native speculative decoding
Vision Tower: 6.03 bpw across 27 ViT blocks
N-Gram Table: 37 GB disk-mapped table (ngram_embedding.safetensors) for fast n-gram lookups without consuming resident VRAM

💻 Quick Start with Pi Code CLI (`pi`)

To use this model directly inside Pi CLI, add the following to your ~/.pi/agent/models.json:

{
  "providers": {
    "vllm-home": {
      "baseUrl": "http://<YOUR_HOST_IP>:8899/v1",
      "api": "openai-completions",
      "compat": {
        "supportsDeveloperRole": false,
        "supportsReasoningEffort": true,
        "thinkingFormat": "qwen-chat-template",
        "chatTemplateKwargs": {
          "enable_thinking": true,
          "preserve_thinking": true
        }
      },
      "models": [
        {
          "id": "Swift1.5-Qwen3.8-Flash-Next-EXL3",
          "name": "Swift1.5 Qwen3.8 Flash Next EXL3",
          "reasoning": true,
          "contextWindow": 1048576,
          "maxTokens": 524288,
          "samplingParams": {
            "temperature": 1.0,
            "top_p": 0.95,
            "top_k": 20
          }
        }
      ]
    }
  }
}

Then run in your terminal:

pi --provider vllm-home --model Swift1.5-Qwen3.8-Flash-Next-EXL3

🚀 Serving with TabbyAPI on NVIDIA DGX Spark

# Recommended environment flags for Grace Blackwell GB10
export EXL3_MOE_MIXEDK_NOSYNC=1
export EXL3_MOE_COOP_MIXEDK=1
export EXL3_MTP_SOLO=1
export PROFILE=concurrent
export MAX_SEQ_LEN=1048576
export CACHE_SIZE=1048576
export NGRAM_RAM=false
export VISION=true
export DRAFT_NUM_TOKENS=5
export HOST="0.0.0.0"
export PORT="8899"
export MODEL_DIR="/path/to/Swift1.5-Qwen3.8-Flash-Next-EXL3"

# Run inference server
python serve.py

Measured Local Performance (GB10 Grace Blackwell)

Generation Speed: ~50–61 tokens/sec
MTP Speculative Acceptance: ~65%
Resident GPU Memory: ~71.9 GB (leaving over 48 GB free for KV cache and 1M tokens!)

📬 Contact, Support & Feedback

☕ Ko-fi Donations: https://ko-fi.com/kattermobile
📧 Support & Bug Reports: support@kattermobile.com
💬 Community Discussions: Feel free to open an issue or discussion thread right here on Hugging Face!

📜 Credits & Licensing

UkisAI: For the distilled Swift 1.5 reasoning foundation (ukisai/Swift1.5-Qwen3.8-Flash-Next).
Qwen Team (Alibaba Cloud): For the monumental Qwen3.8-Flash-Next architecture.
Turboderp: For the incredible ExLlamaV3 engine and EXL3 format.
Unsloth Team: For making efficient large-scale QLoRA fine-tuning possible.

This model follows the Swift Open License v1.0 and Qwen Community License 1.0.

Downloads last month: 1,055

Safetensors

Model size

34B params

Tensor type

BF16

F16

I16