Bonsai-27B-mentria
prism-ml's Bonsai-27B — a natively 1-bit (quantization-aware-trained) 27B model — repacked into mentria-q1g128-v1 safetensors for mentria-engine, a custom WebGPU runtime that runs the model entirely in your browser. 27 billion parameters in 3.79 GB, with an optional vision tower (+741 MB) for image understanding — validated end-to-end in-engine (image → ViT → M-RoPE prefill → answer).
This is the top tier of the mentria model ladder: the Qwen3.5-VL small tiers (0.8B → 2B → 4B) cover phones and laptops; Bonsai-27B serves Apple-Silicon-class GPUs and ≥6 GB discrete GPUs. Same architecture family top to bottom (Bonsai-27B is a 1-bit compression of Qwen3.6-27B — hybrid Gated-DeltaNet + attention with the stock Qwen3-VL vision tower).
Files
| Path | Size | Purpose |
|---|---|---|
bonsai-27b-q1g128-00001-of-00002.safetensors |
~1.9 GB | Language model, shard 1 of 2 |
bonsai-27b-q1g128-00002-of-00002.safetensors |
~1.9 GB | Language model, shard 2 of 2 |
tokenizer.json |
~12 MB | Qwen3.5 BPE tokenizer (248,320 vocab) |
tokenizer_config.json |
~16 KB | Special-token IDs, chat-template metadata |
chat_template.jinja |
~8 KB | Standalone chat template |
bonsai-27b-vl-q4.safetensors |
~741 MB | Vision tower (optional) — 27-layer Qwen3-VL ViT + merger, Q4_0 where 32-block-eligible / F32 elsewhere (the fc2 K=4304 layers) |
preprocessor_config.json |
~400 B | Image preprocessing constants |
Format: mentria-q1g128-v1
Every weight is one bit (0 → −scale, 1 → +scale) with one FP16 scale shared per 128-weight group — ≈1.14 bits/parameter effective. The code words are byte-identical to the upstream GGUF Q1_0 (g128) tensors; only the container changes (safetensors with a scales header, split for the browser's per-buffer limits). Bonsai is QAT — born 1-bit, not post-hoc quantized — so there is no precision loss relative to the published model.
Correctness
Every kernel on every shipping route is teacher-forced-golden-certified against the PrismML llama.cpp reference: 192/192 forced positions, zero decisive disagreements, across 4 engine configurations (including the no-subgroups fallback used by Safari/iOS). Greedy engine output is token-identical to the reference runtime.
Quality (measured s1836 via the PrismML llama.cpp reference runtime — the engine is token-identical to it at greedy)
| Benchmark (protocol) | Score |
|---|---|
| ARC-Challenge (0-shot generative, greedy, thinking off) | 86.6% (259/299) |
| HellaSwag (1000 tasks, 0-shot likelihood acc_norm) | 69.2% |
| Winogrande (1267 tasks, 0-shot likelihood) | 68.9% |
| WikiText-2 perplexity (ctx 2048, lower is better) | 10.99 |
Likelihood-based scores under-report instruction-tuned models relative to generative protocols; use the generative row for cross-model comparisons.
Independent verification of the upstream "≈90% of FP16" claim (2026-08)
We reproduced prism-ml's published 1-bit thinking-mode numbers on this exact repack (their protocol: thinking on, temperature 0.7, top-p 0.95, top-k 20, EvalScope prompt formats; seed-42 subsets; scoring by rule-based answer equivalence + hand review):
| Benchmark (n) | Upstream 1-bit claim | Measured here |
|---|---|---|
| GSM8K (200) | 92.8 | 90.5 ±4.1 |
| MMLU-Redux (228) | 82.75 | 82.9 ±4.9 |
| MATH-500 (120) | 98.0 | 98.1 on completed chains (85.8 under a 3.3× smaller thinking budget) |
A 20-question greedy spot-check through the WebGPU engine matched the reference runtime on every chain that fit the harness context — zero completed-chain divergences. Scope note: the upstream "90%" is an average over 15 benchmarks; retention on agentic and strict instruction-following categories (τ²-Bench, IFBench) is materially lower than on the math/knowledge benchmarks shown here — see the upstream whitepaper before relying on those behaviors.
Performance
| Device | Decode | Notes |
|---|---|---|
| M4 Pro Mac mini, Chrome | ~38-39 tok/s | full fast kernel stack (v14b cooperative-band vecmat) |
| RTX 3060 mobile 6GB, Chrome | ~24 tok/s | fits in 6GB (streaming load, maxSeq 1024) |
Model details
| Base model | prism-ml/Bonsai-27B-gguf |
| Architecture | Qwen3.5-family hybrid: 64 layers — 48 Gated-DeltaNet (16 key / 48 value heads, asymmetric value-side GQA) + 16 full-attention (every 4th layer; 24Q/4KV, head dim 256), hidden 5120 |
| Quantization | mentria-q1g128-v1: 1-bit weights, group size 128, FP16 scale per group (QAT — natively 1-bit) |
| Tokenizer | Qwen3.5 BPE, 248,320 vocab |
| License | Apache 2.0 (upstream prism-ml license preserved) |
About mentria.ai
Mentria is a creative studio for tools, experiments, and visual transmissions. All tools run locally in your browser with zero server dependency.