--- library_name: coreai license: apache-2.0 license_link: https://ai.google.dev/gemma/docs/gemma_4_license base_model: google/gemma-4-E4B-it-qat-q4_0-unquantized tags: - coreai - aimodel - apple-silicon - on-device - gemma-4 - qat - gpu-pipelined pipeline_tag: text-generation base_model_relation: quantized --- Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's `coreai-torch` (LLMs: `coreai.llm.export`) into `.aimodel` bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol ([apple-silicon-llm-bench](https://github.com/john-rocky/apple-silicon-llm-bench), macOS 27 beta 26A5353q, 2026-06-11). This model has no row on [DeviceMark](https://devicemark.github.io/), the on-device LLM leaderboard. # Gemma 4 E4B (text) — Apple Core AI (`.aimodel`) **Gemma 4 E4B's text decoder converted to Apple's Core AI** (the Core ML successor announced at WWDC26), running on iOS 27 / macOS 27 via Apple's `coreai-pipelined` GPU engine — **zero custom kernels, greedy oracle 8/8 exact vs the fp32 Hugging Face reference on the Mac GPU and the iPhone GPU (iPhone is 24/24 token-identical to the Mac on the determinism probe)**. Converted **directly from Google's official QAT release** [google/gemma-4-E4B-it-qat-q4_0-unquantized](https://huggingface.co/google/gemma-4-E4B-it-qat-q4_0-unquantized): bf16 weights **trained for q4_0 rounding**, and q4_0 *is* this bundle's quantization class (per-block-32 absmax linear int4) — Google publishes these checkpoints as "preserving similar quality to bfloat16", so this int4 conversion carries that guarantee **by design**, not by post-hoc gating. > Requires iOS 27 / macOS 27. Conversion code, knowledge base, engine patch stack: > **[coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo)** — > model card: [`zoo/gemma4-e4b.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/zoo/gemma4-e4b.md). ## Use it **New to Core AI? [Start with CoreAIKit 0.7.3](https://github.com/john-rocky/coreai-kit#readme).** Follow its requirements and first-run steps for `qwen3-0.6b`, then open the same release's [ChatDemo](https://github.com/john-rocky/coreai-kit/tree/0.7.3/Examples/ChatDemo). The README records the tested OS/SDK and download size; model and device coverage is stated per example. ⚡ **One line** — run the kit's task op on this model (`import CoreAIOps`; no session, no model plumbing, downloads on first use): ```swift let tldr = try await CoreAI.summarize(text, options: .model("gemma-4-e4b")) ``` Every op, one shape — [Cookbook](https://github.com/john-rocky/coreai-kit/blob/0.7.3/docs/COOKBOOK.md). ▶️ **Run it (source)** — the [ChatDemo runner](https://github.com/john-rocky/coreai-kit/tree/0.7.3/Examples/ChatDemo) (GUI + CLI, one app for every chat model in the catalog): ```bash git clone --branch 0.7.3 --depth 1 https://github.com/john-rocky/coreai-kit export DEVELOPER_DIR=/Applications/Xcode-27.0.0-RC.app/Contents/Developer open -a /Applications/Xcode-27.0.0-RC.app coreai-kit/Examples/ChatDemo/ChatDemo.xcodeproj # → Run, then pick "Gemma 4 E4B" in the model picker # agents / headless (macOS): cd coreai-kit/Examples/ChatDemo swift run -c release chat-cli --model gemma-4-e4b --prompt "What can you do, offline?" ``` Use Xcode build **27A266a** from the release's `.xcode-pin`; adjust the app path if your installation is named differently. 💻 **Build with it** — complete; the glue is kit API, copy-paste runs: ```swift import CoreAIKit let id = "gemma-4-e4b" let chat: ChatSession if id == "qwen3-0.6b" { // Freeze the release starter; other selections retain the live catalog's // model-specific dispatch (including paired Gemma bundles). guard let model = ModelCatalog.builtin.entry(id: id)?.modelID else { throw CoreAIKitError.modelNotAvailableOnPlatform(id: id) } chat = try await ChatSession(model: model) } else { chat = try await ChatSession(catalog: "gemma-4-e4b") } let reply = try await chat.respond(to: prompt) // reply: the answer, generated fully on-device ``` **When Apple's FoundationModels built-in model isn't enough, keep your session code and swap the model — one line.** CoreAIKit's [`KitGemmaModel`](https://github.com/john-rocky/coreai-kit#works-with-apples-foundationmodels-api) plugs supported chat bundles into the system `LanguageModelSession`. Tool calling depends on the model's dialect; guided generation also requires a compatible sequential engine. Check the linked support matrix before using either capability. The take-home is [`Examples/ChatDemo/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/0.7.3/Examples/ChatDemo/Sources/QuickStart.swift) — this exact code as one typed function, no UI; the CLI is an argument shell over it, and the GUI drives the same `ChatSession` across turns for its transcript. Multi-turn? Hold the `ChatSession` and call `respond(to:)` per turn — it keeps the conversation history; `streamResponse(to:)` yields tokens as they decode. **Integration checklist** - SPM: `https://github.com/john-rocky/coreai-kit` (exact **0.7.3**) → product **CoreAIKit** - Info.plist: none needed - Entitlements: none needed (macOS) - First run downloads the model — ~7,590 MB (Mac) — then it loads from the local cache (Application Support; progress via the `downloadProgress` callback) - Measure in Release — Debug is ~3× slower on per-token host work ## Measured (greedy; M4 Max / iPhone 17 Pro, settled device) | config | files | size | M4 Max decode / prefill | iPhone decode / prefill | |---|---|---|---|---| | ★ **provider** (runs BOTH platforms) | `gpu-pipelined/gemma4_e4b_qat_decode_int4lin/` + `ios-frontend/gemma4_e4b_qat_gather_raw/` | 3.7 + 3.4 GB | 53.2 / 62.6 | **15.1 / 21.3** | | ★ **provider, iPhone-ready AOT** | `gpu-pipelined/gemma4_e4b_qat_decode_int4lin_aotc_h18p/` (precompiled `.aimodelc`, **h18p = iPhone 17 Pro class only**) + the same tables | 3.7 + 3.4 GB | — | same as above — skip the AOT step | | **tbl** (Mac-fastest) | `gpu-pipelined/gemma4_e4b_qat_decode_int4lin_tbl/` + the two `embed_per_layer.*` table files | 3.7 + 2.7 GB | **55.8** / 61.0 | not viable (3.7 GB graph + 2.7 GB owned tables > the ~6.4 GB entitled limit) | On iPhone the working set stays tiny — measured peak footprint **2.2 GB** (4.2 GB headroom): the PLE table rides as a clean mmap and the AOT executable pages are evictable. Both phases land exactly on the bandwidth model (~2.1 GB int4/token). ## What E4B is (config + checkpoint verified) Clean **dense** model — no MoE. 42 layers (full attention every 6th), hidden 2560, intermediate 10240 uniform, 8 query heads / **2 KV heads**, dual head_dim 256/512, 18 KV-shared layers (the engine bundle stacks the 24 non-shared layers into ONE unified padded KV pair), per-layer embeddings (the [262144, 10752] int8 table ships in `ios-frontend/gemma4_e4b_qat_gather_raw/`), final-logit softcap 30. The QAT checkpoint prunes the never-used KV projections on the shared layers — the zoo's loader handles both layouts. ## Run contract (each item is load-bearing) Full story + traps: [pipelined-engine page](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/pipelined-engine.md). 1. Swift stack = `apple/coreai-models` + the zoo's patch stack ([`apps/*.patch`](https://github.com/john-rocky/coreai-model-zoo/tree/main/apps), in order). The ★ provider bundle needs `EngineOptions.perTokenInputProvider` (`coreai-pipelined-per-token-inputs.patch`); the tbl bundle needs `EngineOptions.staticInputBuffers` (`coreai-pipelined-static-inputs.patch`). 2. Provider mode: per token, fill `ple_tokens [1,1,42,256]` fp16 from the table dump — `row = i8[id] * scale[id] * sqrt(256)`, mmap-gathered (~0.1 ms). tbl mode: bind `ple_table` ← `embed_per_layer.i8` and `ple_scale` ← `embed_per_layer.scale.f32` as **OWNED `storageModeShared` MTLBuffers** (buffer-backing traps in the knowledge page). 3. `COREAI_CHUNK_THRESHOLD=1` **before** engine creation; **never call `engine.warmup()`** (S=1 graph; a 1-token generate after load is the warmup). 4. iPhone: **AOT is mandatory** (the 3.7 GB-constants graph crashes the on-device specializer) — use the precompiled `_aotc_h18p/` bundle, or `xcrun coreai-build compile .aimodel --platform iOS --preferred-compute gpu --architecture h18p --expect-frequent-reshapes` and point `metadata.json`'s `assets.main` at the `.aimodelc`. Ship the `com.apple.developer.kernel.increased-memory-limit` entitlement as headroom insurance, and bench a **settled** device (a just-unlocked iPhone under-reads ~35%). Reproduce from scratch (oracle + tables are checkpoint-derived — regenerate for any new weights): [`conversion/export_gemma4_decode_pipelined.py`](https://github.com/john-rocky/coreai-model-zoo/blob/main/conversion/export_gemma4_decode_pipelined.py) with `--hf-id google/gemma-4-E4B-it-qat-q4_0-unquantized`. ## License Gemma 4 is released under the Apache License 2.0 — see the base card [google/gemma-4-E4B-it-qat-q4_0-unquantized](https://huggingface.co/google/gemma-4-E4B-it-qat-q4_0-unquantized) and Google's [Gemma 4 license page](https://ai.google.dev/gemma/docs/gemma_4_license). These `.aimodel` bundles redistribute the weights in a different serialization; the conversion adds no additional restrictions. Sibling repo (E2B, incl. its own official-QAT bundles): [gemma-4-E2B-CoreAI](https://huggingface.co/mlboydaisuke/gemma-4-E2B-CoreAI). --- **More models in this format:** [Core AI Model Zoo](https://huggingface.co/collections/mlboydaisuke/core-ai-model-zoo-6a7ff330f753e8dcae04671a) — 75 models, each with the recipe that produced it. **Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.