StartLux-Decision-v1.1-4B-Instant-MLX-8bit

StartLux-Decision-v1.1-4B-Instant as an 8-bit MLX build for Apple silicon. StartLux-Decision-v1.1-4B-Instant is a 4B decision model. Given a state (text or JSON) and typed questions about it (pick an option, yes or no, or a level on a scale of 2 to 10), it returns a probability for every option of every question from one forward pass, without generating text. Requests and responses use the TypeSafe /v1/systemone format.

Formats: BF16 · GGUF · MLX 8-bit · MLX 4-bit · Code: StartLuxLabs/StartLux-Decision · Paper: arXiv 2610.17309

Decision Index 0.3, public subset, as of 11 October 2026: the StartLux-Decision Instant models against other decision models of each size. Figure 1 of the technical report.

This build

Converted from the BF16 weights with mlx-vlm 0.7.6 (python -m mlx_vlm convert -q --q-bits 8 --q-group-size 64). The text model has affine 8-bit weights in groups of 64; the vision tower stays in BF16 and is not quantized, so the folder reads images as well as text. 5.15 GB in total, of which 0.66 GB is the vision tower. The multi-token-prediction layer is left out. The text weights are byte-identical to our earlier text-only build. decision_config.json (option letters, read-out temperatures) is the BF16 model's, and the tokenizer gives the same token ids.

Instant models read their prompt bidirectionally in the full-attention layers (one in four; the linear-attention layers stay causal), and config.json keeps the flag "jev_prompt_attention": "bidirectional". Plain mlx-vlm or mlx-lm ignore the flag and read every prompt causally, which makes it a different model: read that way, our 0.8B and 4B Instant models chose the same option as BF16 served by vLLM for 96.5% and 95.5% of the questions, against 99.6% and 99.9% with the bidirectional prompt. Serve this build with an engine that reads the prompt bidirectionally. The MLX backend of startlux-decide applies this mask and checks at load time that it is on; a version without it refuses the folder instead of reading it causally.

Run

git clone https://github.com/StartLuxLabs/StartLux-Decision
cd StartLux-Decision && pip install -e '.[mlx]'          # the package and mlx-vlm, on Apple silicon
hf download startlux-models/StartLux-Decision-v1.1-4B-Instant-MLX-8bit --local-dir ~/models/StartLux-Decision-v1.1-4B-Instant-MLX-8bit
startlux-decide serve --model ~/models/StartLux-Decision-v1.1-4B-Instant-MLX-8bit --backend mlx --port 8090

curl -s localhost:8090/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": {"ticket": "I was charged twice for order #4411 and the app still shows it as unpaid."},
  "questions": {
    "team": {"type": "choice", "instructions": "Which team should handle this ticket?",
             "criteria": {"billing": "Payments and refunds", "shipping": "Delivery", "technical": "App and login"}},
    "urgent": {"type": "noul", "instructions": "Should this ticket be answered today?"}
  }
}'

Every question comes back with a probability for each option. The server reads text and up to 8 images per request, each resized to at most 1,048,576 pixels (images: base64 strings or data URIs, next to state and questions). A request that asks this model to think gets status 422. Loaded with plain mlx-vlm or mlx-lm the folder only generates text: the option-letter read-out and its temperatures are in the package.

Agreement with BF16

Before release we read 100 questions of Decision Index requests (92 with prompts under 1,000 tokens, 8 with 1,000 to 4,000 tokens) in one pass with this build and with the BF16 weights, both in MLX. This build gave the same answer to 98 of the 100. Per question, the largest change of an option's probability was 0.006 on average and 0.06 at most. Of the two answers it changed, one was a tie in BF16 (equal top probabilities) and the other a near tie in BF16 (the top two options 0.028 apart). This comparison was made with the earlier text-only build of this model, whose text weights are byte-identical to this one's.

In an earlier study against BF16 served by vLLM on an H200, on 700 to 800 questions per build, 8-bit builds of our 0.8B, 2B and 4B Instant models chose the same option for 99.5% to 99.6% of the questions, and 4-bit builds (0.8B and 4B Instant, 4B AutoThink) for 96.3% to 97.5%.

Speed

On a MacBook Pro with an Apple M5 Pro (20-core GPU, 64 GB of memory) and mlx 0.32.3, one read-out of a prompt of 1,024 tokens takes 602 ms with this build (BF16: 522 ms), and one of 256 tokens 95 ms. These timings are from the earlier text-only build of this model, whose text weights are byte-identical to this one's.

License

This is a quantized version of StartLux-Decision-v1.1-4B-Instant. The weights are released under CC BY-NC 4.0, as the BF16 weights are: free for research and other non-commercial use, with attribution. Commercial use requires a separate license from StartLux Labs (contact@startlux.com). The inference package startlux-decide is Apache-2.0. See LICENSE and NOTICE.