eduardoworrel/alias3-micro-1026-ONNX-decode-only

alias3-micro-1026 · ONNX decode-only (token-by-token, 7 dtypes)

Decode-only ONNX graphs of aliasrobotics/alias3-micro-1026: one token per call, hidden state exposed for System 1 heads, driven by ct0-edge in the browser or Node. Browser: onnx/model_q4.onnx (3.03 GB, onnxruntime-web WebGPU; ct0-edge default) — severity levels 26/50 = fp32, Chrome decisions = onnxruntime CUDA on every decision-test question. Base model: aliasrobotics/alias3-micro-1026 (Alias Robotics), whose card labels the weights a “research preview”. Paper: arXiv:2610.16550. System 1 / System 2 and the bilinear pointer-head interface come from that paper, which draws on Kahneman's dual-process framing and adapts the typed-decision head from the Jev / System One lineage of TypeSafe AI (pointer networks, Vinyals et al., 2015).

ONNX · transformers.js | ONNX · GenAI | ONNX · decode-only | MLC q4f32_1 | MLC q4f16_1 | LiteRT.js

Score vs size

Produced by: fp32 — optimum export + our optimum-onnx patch · uint8 — optimum ORTQuantizer as-is · fp16 · q8 · q4 · q4f16 · bnb4 — our scripts.

System 1 per file

Accuracy is not statistically distinguishable from the original (0 of 70 paired tests, McNemar, before or after Holm correction); individual decisions change on 0.6–17.4% of questions; the test set (50–65 questions per mode) only detects large losses.

Dark segment: accuracy when the answer is read straight from the model's logits (the Yes/No, option or digit tokens). Lighter segment: the extra accuracy when the readout head (head.bin, or head_score.bin for severity) reads the hidden state instead. Severity (head_score.bin) is a proof of concept: fitted on 30 synthetic cases; using it as a severity scorer calls for a larger, independently labelled set.

System 1 table on the decision test set (logits / head per mode)

Status vs the original model: 🟢 within 2 answers of the original (or better) · 🟡 3+ below the original, not significant · 🔴 significantly below the original (McNemar p < 0.05) · ⚪ original model (PyTorch bf16), the reference.

file	yes/no (65) · logits	yes/no (65) · head.bin	multiple choice (57) · logits	multiple choice (57) · head.bin	severity (50) exact · logits	severity (50) exact · head_score.bin	severity (50) ±1 · head_score.bin	changed answers vs original (default read)
Original (PyTorch bf16)	⚪ 54/65	⚪ 54/65	⚪ 39/57	⚪ 46/57	⚪ 20/50	⚪ 26/50	⚪ 47/50	—
model_q4f16	🟢 53/65	🟢 54/65	🟢 38/57	🟢 46/57	🟢 19/50	🟢 24/50	🟢 50/50	16.3%
model_q4	🟢 55/65	🟢 54/65	🟢 39/57	🟢 48/57	🟢 20/50	🟢 26/50	🟢 47/50	17.4%
model_fp16	🟢 54/65	🟢 54/65	🟢 39/57	🟢 47/57	🟢 20/50	🟢 27/50	🟢 47/50	1.2%
model	🟢 54/65	🟢 54/65	🟢 39/57	🟢 47/57	🟢 20/50	🟢 26/50	🟢 47/50	0.6%
model_bnb4	🟢 55/65	🟢 55/65	🟢 44/57	🟢 47/57	🟢 20/50	🟢 30/50	🟢 46/50	12.2%
model_quantized	🟢 54/65	🟢 55/65	🟢 40/57	🟢 44/57	🟢 21/50	🟢 27/50	🟢 46/50	2.9%
model_uint8	🟢 55/65	🟡 51/65	🟢 37/57	🟢 46/57	🟢 19/50	🟡 22/50	🟢 47/50	14.0%

Files: onnxruntime CUDA (uint8: CPU), token-by-token.

Browser details (load, timing, agreement)

file	size	fits browser	load (Chrome)	time per decision	Chrome decisions = server
model_q4f16.onnx	2.73 GB	✅	12.0 s	15.4 s (median per probe, token-by-token)	394/394 vs onnxruntime CUDA
model_q4.onnx	3.03 GB	✅	12.3–34.1 s	20.7 s (median per probe, token-by-token)	394/394 vs onnxruntime CUDA
model_bnb4.onnx	2.73 GB	✅	10.0–11.5 s	29.5 s (median per probe, token-by-token)	394/394 vs onnxruntime CUDA
model.onnx	19.37 GB	❌ > 4 GB	—	—	—
model_fp16.onnx	8.42 GB	❌ > 4 GB	—	—	—
model_quantized.onnx	4.91 GB	❌ > 4 GB	—	—	—
model_uint8.onnx	4.85 GB	❌ > 4 GB	—	—	—

Chrome WebGPU, Apple M2 Pro, onnxruntime-web 1.30.0 via ct0-edge.

Files: size, tool, runtime

file	dtype	size	external data	produced by	runtime (measured)
onnx/model_q4f16.onnx	q4f16	2.73 GB	1 file	S · mkq4f16.py (MatMulNBits, fp16 scales)	onnxruntime-web 1.30 WebGPU (Chrome) · onnxruntime CUDA
onnx/model_q4.onnx ⭐	q4	3.03 GB	1 file	S · mkvar.py (MatMulNBits int4 + GBQ int4 embedding)	onnxruntime-web 1.30 WebGPU (Chrome) · onnxruntime CUDA
onnx/model_fp16.onnx	fp16	8.42 GB	1 file	S · mk_fp16full.py (all weight MatMuls fp16)	onnxruntime CUDA
onnx/model.onnx	fp32	19.37 GB	1 file	O+P · optimum export (jit.trace) + our optimum-onnx patch	onnxruntime CUDA
onnx/model_bnb4.onnx	bnb4	2.73 GB	1 file	S · mk_bnb4_decode.py (MatMulBnb4 NF4 + GBQ uint4 embedding)	onnxruntime-web 1.30 WebGPU (Chrome) · onnxruntime CUDA
onnx/model_quantized.onnx	q8	4.91 GB	1 file	S · mkvar.py (QDQ W8 + GBQ uint8 embedding)	onnxruntime CUDA
onnx/model_uint8.onnx	uint8	4.85 GB	1 file	O · optimum ORTQuantizer QUInt8	onnxruntime CPU

Provenance: O = official tool as-is · O+P = official tool + our patch or wrapper · S = our script (what the official tool does not do). ⭐ = recommended for the browser (ct0-edge fallback). Also: config.json, generation_config.json, tokenizer, heads/, SHA256SUMS.txt.

Generation (System 2)

System 1 = single-forward typed decisions through the bilinear pointer head (sections above); System 2 = generation, the usual decode loop from the same weights. Terms as in the base model card and the paper.

96-step greedy, 22-token prompt. Reference = the decode-only fp32 ONNX tokens on onnxruntime CPU (method of record; there is no bf16 original generation run).

Generation details per file

file	test	free-running = fp32	first divergence	teacher-forced	notes
model.onnx	96-step greedy	96/96	—	96/96	teacher-forced cos mean 1.0000
model_fp16.onnx	96-step greedy	96/96	—	96/96	teacher-forced cos mean 1.0000
model_quantized.onnx	96-step greedy	94/96	94	95/96	teacher-forced cos mean 0.9978
model_uint8.onnx	96-step greedy	89/96	81	95/96	teacher-forced cos mean 0.9847
model_bnb4.onnx	96-step greedy	5/96	2	93/96	teacher-forced cos mean 0.9706
model_q4.onnx	96-step greedy	2/96	2	93/96	teacher-forced cos mean 0.9641
model_q4f16.onnx	96-step greedy	2/96	2	87/96	teacher-forced cos mean 0.9602

Reference: model.onnx (fp32) on onnxruntime CPU, 22-token prompt. Every 4-bit file diverges at step 2 (a near-tie after <think>).

Peculiarities

• graph: one token per call, no attention_mask; a whole-prompt call gives logits cos 0.012
• inputs / outputs: 65 inputs (input_ids + past_* caches) · 66 outputs (logits, present_*, hidden)
• head outputs exposed: hidden name: /model/norm/Mul_1_output_0 (all 7 files) — listed in onnx/HIDDEN_OUTPUTS.json
• cache dtype: float32 for every file, fp16 included
• single-file vs 2-chunk external data: single file for every dtype (model_X.onnx_data)
• needs onnxruntime-web ≥ 1.30: yes (GatherBlockQuantized, MatMulNBits, MatMulBnb4); measured on 1.30.0
• bnb4: NF4 (MatMulBnb4, block 64) + uint4 embedding
• transformers.js generate(): not supported (use the optimum/Scan graph repo)
• ct0-edge prompt cap: maxSafeTokens 3600 on WebGPU, 7200 on Node CPU (measured)
• long context (one text, 7 positions): fp32 cos ≥ 0.9973 up to 5990 tokens, argmax 7/7
• export of the model (optimum + our optimum-onnx patch): optimum-onnx has no qwen3_5 export config, and the stock Gated-DeltaNet forward does not trace. We run optimum's onnx_export_from_model on the jit.trace path with a one-token dummy input and the attention_mask input removed, plus our optimum-onnx patch (qwen3_5 config, hybrid cache, export-safe Gated-DeltaNet forward). This graph uses optimum's own jit.trace export path (one-token dummy input, attention_mask input removed), with our optimum-onnx patch adding the qwen3_5 config, cache and forward. Effect: one token per call, no Scan (11,434 nodes, 19.37 GB in fp32); the cache goes in as past_* inputs.
• post-final-norm hidden output: the stock export returns logits and the cache only; this graph carries the post-final-norm hidden state as an extra output (66 outputs), which the System 1 heads read.
• no position_ids input: the graph is built without a position_ids input on purpose. In transformers.js the get_rope_index path is taken only when the graph exports that input, and that function reads vision_config, which a text-only Qwen3.5 config does not have.
• fp16 (our script): optimum's fp16 optimizer is not available for qwen3_5_text, and converting every fp32 cast to fp16 removes the precision islands around the norms and the Gated-DeltaNet state math (cos 0.913 measured). Our script wraps only the weight matmuls in fp16 casts and keeps activations, norms, the recurrence state and the inputs and outputs in fp32; the embedding table is stored in fp16 and the output matmul reuses it (tied), so there is no second copy. Measured over 96 greedy steps against fp32: 96/96, cos ≥ 0.99999.
• q8 (our script) and uint8 (optimum as-is): In our run ONNX Runtime's dynamic quantizer loaded the whole 19 GB graph and left about 19 GB of temporary files, so model_quantized.onnx comes from a streaming script of ours: int8 per-channel weights as QDQ, and a uint8 GatherBlockQuantized embedding because that op does not accept int8. model_uint8.onnx is the exception: optimum's ORTQuantizer with its stock dynamic per-channel config and QUInt8 weights.
• 4-bit files (our scripts): optimum has no 4-bit path. Our scripts read the fp32 graph tensor by tensor: q4 = MatMulNBits int4 block 32 and a GatherBlockQuantized int4 embedding; q4f16 = the same with fp16 scales between fp16 casts; bnb4 = MatMulBnb4 NF4 block 64 and a uint4 embedding. The bnb4 file replaces the NF4 file written by the transformers.js quantize.py, which was 4.9 GB with an fp32 embedding.
• ours vs the official tool — q8: over 96 greedy steps against fp32, our weights-only q8 matches 94/96 (first divergence at step 94, teacher-forced 95/96); optimum's dynamic ORTQuantizer q8, built on the same graph, matches 74/96 (first divergence at step 74, teacher-forced 94/96). Ours is better here.
• ours vs the official tool — uint8 (the official tool won): our own uint8 script fixed the zero point at 128 and cut half the range (cos 0.255), while optimum's ORTQuantizer matched 89/96 greedy steps (teacher-forced 95/96). So model_uint8.onnx is optimum's file as it came out.
• ours vs the official tool — q4 and bnb4: the transformers.js quantize.py files, scored the same way over 96 steps: q4 is 5.18 GB (the embedding stays fp32) against 3.03 GB for ours, and both diverge from fp32 at step 2 and agree teacher-forced on 93/96; bnb4 is 4.91 GB against 2.73 GB for our NF4 file, both 5/96 autonomous, teacher-forced 94/96 against 93/96, and on the decision test set yes/no logits 55/65 for both with multiple-choice logits 42/57 (official) against 44/57 (ours). Same quality at 41–44% smaller, so our files are the ones shipped.
• ours vs the official tool — q4f16: the q4f16 file written by quantize.py does not load in ONNX Runtime 1.31 (CPU or CUDA): its fp16 conversion does not keep the fp32 islands around RMSNorm and an Add node mixes float and float16. We did not repair it by hand; q4f16 here is our file.
• System 1 head file (head.bin recovered from a truncated release file) and prompt markers: the official head.pt is a truncated streaming zip and cannot be loaded, so head.bin is recovered from it (q exact; k.weight 89% of rows; k.bias lost and estimated as one offset). The marker-to-id mapping is not released, so the prompt markers are plain text, and the official System 1 endpoint needs a fork of SGLang (--systemone-head) that is not in public SGLang. Table and numbers: “Changes to the official head” below.

Our patches and scripts are kept by the authors and are available on request.

Usage

Browser or Node, ct0-edge:

npm i ct0-edge @huggingface/transformers "onnxruntime-web@^1.30"

import { CT0 } from "ct0-edge";

const ct0 = await CT0.load({
  runtime: "decode-only",                    // "auto" uses this repo when genai/WebGPU is unavailable
  device: "webgpu",                          // "wasm" without WebGPU; "cpu" / "cuda" in Node
  repo: "eduardoworrel/alias3-micro-1026-ONNX-decode-only",
  decodeModel: "onnx/model_q4.onnx",         // or model_q4f16.onnx / model_bnb4.onnx; data: <model>_data
  heads: true,                               // heads/head.bin + heads/head_score.bin
});
await ct0.ask({ kind: "score", state: "..." });   // { type: "score", level, raw }

Python, one token per call (the loop that produced the numbers above):

import numpy as np, onnxruntime as ort
s = ort.InferenceSession("onnx/model_q4.onnx", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
outs = [o.name for o in s.get_outputs()]
st = {i.name: np.zeros([d if isinstance(d, int) else 1 for d in i.shape], np.float32)
      for i in s.get_inputs() if i.name != "input_ids"}
for tid in prompt_ids:                                   # one token per call, prompt included
    o = dict(zip(outs, s.run(None, {**st, "input_ids": np.array([[tid]], np.int64)})))
    hidden = o["/model/norm/Mul_1_output_0"][0, 0]       # keep it at <decide> and at the last position
    for L in range(32):
        for a, b in (("present_conv", "past_conv"), ("present_recurrent", "past_recurrent")):
            if f"{a}.{L}" in o: st[f"{b}.{L}"] = o[f"{a}.{L}"]
        for k in ("key", "value"):
            if f"present.{L}.{k}" in o: st[f"past_key_values.{L}.{k}"] = o[f"present.{L}.{k}"]

Heads

file	size	read by	shipped
heads/head.bin	2,622,548 B	yes/no, multiple choice — bilinear pointer head over the hidden at <decide> and at the last / option positions	✅ in this repo
heads/head_score.bin	2,622,668 B	severity (levels 1–5) — ridge probe on the last-position hidden, same file format (proof of concept: fitted on 30 synthetic cases; using it as a severity scorer calls for a larger, independently labelled set.)	✅ in this repo

mode	ct0-edge default read	alternative
yes/no	logits Yes vs No (no head)	head.bin
multiple choice (1 of K)	head.bin	logits of each option's first token
severity (1–5)	head_score.bin	logits of digits 1…5

Hidden positions: <decide> (the final > of its 4 tokens >< dec ide >) and the last prompt position (choice: each option's position).

Changes to the official head

Each decision is read two ways from one prefill: by the logits at the last prompt position (no extra weights) and by the head on the hidden states at <decide> and the key positions (the rows after the first).

	official (base card / paper)	ours	why
logits read	not part of the official System 1 (the paper leaves head-vs-logits to future work)	compare next-token logits: Yes vs No; first token of each option; digits 1–5	weight-free baseline every runtime supports; the dark segment in the System 1 figure
weights file	head.pt on the base repo: truncated streaming zip (no central directory), cut inside k.weight	head.bin, recovered from it: q.weight/q.bias 100% (CRC match), k.weight 89% (227/256 rows), k.bias lost → one score offset (+16.27) estimated without labels	the official file cannot be loaded
format	PyTorch head.pt for the CSI SGLang fork (--systemone-head)	head.bin (fp16, JSON header) read by ct0-edge systemone.ts in JS, same math	on-device runtimes
yes/no read	bilinear pointer head with K = 2 options	query at <decide>, key at the last position, plus the offset	marker-to-id mapping and prompt row are not published
severity read	ordinal scale as a K-option pointer (System 1 severity 44.4% in the paper)	new head_score.bin: ridge probe on the last-position hidden (λ = 10), encoded exactly in head.bin's 2-row layout; trained on 30 labelled cases, measured on the 50 severity questions (proof of concept: fitted on 30 synthetic cases; using it as a severity scorer calls for a larger, independently labelled set.)	the released head does not order severity
prompt markers	reserved ids (mapping unpublished)	plain-text markers	see the note below the table
temperature / orientation	T = 2.0; query at <decide>, keys at </opt>	kept as released	—

Prompt markers: the paper maps the markers (<state>, <q>, <opt>, </opt>, <decide>) onto reserved token ids. The model has 243 unnamed embedding slots (ids 248077–248319) beyond the tokenizer's last token (248076), but the marker-to-id mapping isn't published, so we write the markers as plain text, which the tokenizer splits into ordinary tokens.

“Extra from the head” in the System 1 figure = how many more questions the head read gets right than the logits read on the same decision test set (ours).

How measured

• Decision test set (ours): 172 security questions with known answers, written by us (not part of the CT0 release): 65 yes/no, 57 multiple choice (chance 0.272), 50 severity (1–5, held out from head_score.bin); sha256 88e0f6c5…
• Stats: 95% bootstrap CI (2000 resamples) · McNemar exact, paired per question, vs the original model (PyTorch bf16, revision 61c0e710)
• Hardware: server rows: onnxruntime CUDA EP on RunPod GPUs · browser rows: Chrome 155 WebGPU, Apple M2 Pro
• Files: eduardoworrel/alias3-micro-1026-ONNX-decode-only · base model: aliasrobotics/alias3-micro-1026

Citation

If you use these files, please cite the base model:

@article{ct0_2026,
  title   = {{CT0}: Cybersecurity Transformer},
  author  = {Mayoral-Vilches, V{\'i}ctor and {Alias Robotics}},
  journal = {arXiv preprint arXiv:2610.16550},
  year    = {2026},
  url     = {https://arxiv.org/abs/2610.16550}
}

Base model card: aliasrobotics/alias3-micro-1026.

License: Apache-2.0 (weights, as upstream).