litert-community/iris-3b-Turbo-LiteRT

iris-3b Turbo — LiteRT

This repository is a conversion of iris-3b Turbo (Speridlabs, Apache-2.0) to LiteRT, made with litert-torch 0.9.4 and ai-edge-quantizer 0.9.0 for LiteRT 2.2.0 CompiledModel: on a Galaxy S26 it makes a 768×768 image from a text prompt in 130.4 s (the mean of two images) with every graph on the phone's GPU, the transformer's trunk as int8 weights, and the trunk and the pixel stage at the GPU's default precision (iris3b_config_dit_int8_fp16.json), against 232.6 s (two images) with every graph at FP32 (iris3b_config.json), the two settings taking turns in one session on the phone (measured 2026-10-31).

Nothing leaves the phone: the tokenizer, the Qwen3-VL-4B text encoder, the 3B pixel-space diffusion transformer and the sampler all run on it. iris-3b draws pixels directly (no VAE), and Turbo needs 4 steps with no classifier-free guidance.

Every image above was made on the Pixel 8a, with every graph on its GPU at FP32.

How it runs

prompt -> Qwen3-VL tokenizer -> bf16 embedding lookup                                   (host)
       -> 12 text encoder graphs, one per hidden-state tap (decoder layers 2, 5, ..., 35)
       -> text adapter graph (once per prompt)
       -> 4 steps x (3 dual-stream graphs -> 3 single-stream graphs -> pixel-stage graph)
          -> sampler update                                                              (host)
       -> 768x768 RGB

Each graph is loaded for its run and closed again, so one graph's weights are in memory at a time (the largest graph file is 0.9 GB; the app's peak memory is 4.6–5.0 GiB with the FP32 setting and 3.2 GiB with the int8 setting on the Galaxy S26, see Measured). On both phones below, every graph compiles whole onto the GPU delegate (Replacing N out of N node(s) with delegate (LITERT_CL) node, yielding 1 partitions; the int8 graphs: checked on the Galaxy S26). The fully-connected weights are stored as fp16 with a DEQUANTIZE in front of each; activations are fp32, and the GPU runs at FP32. The int8 transformer setting (iris3b_config_dit_int8_fp16.json) swaps the six trunk graphs (the dual- and single-stream blocks) for versions with int8 weights and runs them and the pixel stage at the GPU's default precision (CompiledModel.GpuOptions.Precision.DEFAULT in LiteRT's Kotlin API, fp16 arithmetic; the FP32 setting uses Precision.FP32); the text encoder and the adapter stay at FP32.

Minimal usage

model/ holds the 24 files both pipelines read (13.7 GB): 20 graphs, the token embedding table, the host tensors, the tokenizer tables and iris3b_config.json. It also holds the int8 transformer setting: iris3b_config_dit_int8_fp16.json and its six iris_dit_*_i8w_dit.tflite graphs (2.43 GB), which take the place of the six _f16w_ trunk graphs (11.3 GB for the 24 files of that setting). The Android code reads the config named iris3b_config.json, so that setting's config goes onto the phone under that name (the sample app's install script does this with CONFIG=iris3b_config_dit_int8_fp16.json). The Python pipeline below runs the FP32 setting.

Python (CPU)

python/iris3b_litert.py is the whole pipeline in one file: the tokenizer, the embedding lookup and every host step, with each graph on LiteRT CompiledModel (CPU, XNNPACK). It runs the same files and the same steps as the Android engine, so it is the reference to check a port against.

hf download litert-community/iris-3b-Turbo-LiteRT --local-dir iris-3b-Turbo-LiteRT
cd iris-3b-Turbo-LiteRT/python
pip install -r requirements.txt
python iris3b_litert.py --prompt "a cozy wooden cabin in a snowy pine forest at night, warm light in the windows" --seed 7 --out cabin.png

From Python:

from iris3b_litert import Iris3B, save_png, seed_noise

pipe = Iris3B("../model", threads=8)  # reads model/iris3b_config.json; each graph is loaded for its run
image = pipe.generate("a red apple on a wooden table, studio lighting", seed_noise(0))
save_png(image, "apple.png")          # image: float32 [3, 768, 768], clamped to [-1, 1]

seed_noise(seed) is java.util.Random(seed).nextGaussian(), the starting noise the Android code below makes for the same seed (checked bit for bit against the JVM's java.util.Random for seeds 0 and 7). --reference-noise starts from the noise of the upstream PyTorch reference run instead. With --reference-noise, the apple prompt took 23.2 s on a Mac (Apple M4 Max, 8 CPU threads), and its image matches the PyTorch fp32 image at PSNR 106.45 dB / SSIM 1.0.

Kotlin (Android, GPU)

android/ is the engine of the iris-3b sample app: 7 Kotlin files in package com.iris3b that use only the JDK and LiteRT. Add implementation("com.google.ai.edge.litert:litert:2.2.0"), copy the files into your app, and put one setting's 24 files from model/ in the app's files folder (by default the FP32 setting's: iris3b_config.json and the 23 files it names; android/README.md shows how). Then:

import android.graphics.Bitmap
import com.google.ai.edge.litert.Environment
import com.iris3b.HostOps
import com.iris3b.IrisPipeline
import com.iris3b.LiteRtExec
import com.iris3b.PromptTokens
import com.iris3b.RunFiles
import com.iris3b.StageLog
import java.io.File
import java.util.Random

// One setting's 24 files from model/ (by default the FP32 setting's) are in the app's files folder. Run this off the main thread (minutes).
val dir = context.filesDir
File(dir, "iris3b_rows.json")
    .writeText("""{"kind":"generic","config":"iris3b_config.json","inputs":[],"rows":[]}""")
val files = RunFiles(dir, "iris3b_rows.json")  // reads iris3b_config.json: graphs, GPU FP32, host tensors
files.materializeTokenizer()                    // writes tok/: vocab, merges, template ids
val tokens = PromptTokens(File(dir, "tok"))
val env = Environment.create(context)           // one Environment for every graph of the process
val pipeline = IrisPipeline(LiteRtExec(env, dir), files.engineConfig(), files.host, tokens)

val random = Random(7L)                         // the starting noise of seed 7
val noise = FloatArray(HostOps.IMAGE_FLOATS) { random.nextGaussian().toFloat() }
val log = StageLog(clock = System::nanoTime, memory = { emptyMap() })
val result = pipeline.generate(
    "a cozy wooden cabin in a snowy pine forest at night, warm light in the windows", noise, log)

// result.image: CHW floats [3, 768, 768] in [-1, 1] -> an ARGB bitmap
val size = HostOps.IMAGE_SIZE
val plane = size * size
fun level(v: Float) = ((v + 1f) * 127.5f + 0.5f).toInt().coerceIn(0, 255)
val pixels = IntArray(plane) { i ->
    (0xFF shl 24) or (level(result.image[i]) shl 16) or
        (level(result.image[plane + i]) shl 8) or level(result.image[2 * plane + i])
}
val bitmap = Bitmap.createBitmap(pixels, size, size, Bitmap.Config.ARGB_8888)
tokens.close()
env.close()

IrisPipeline also takes stop, read before every graph (the sample app's Stop button and its free-memory guard both go through it), and StageLog takes a listener that sees every step as it happens.

Demo

The sample app on a Pixel 8a: the prompt in the editor, Generate, the image. The middle is sped up; the phone took 508.3 s (the app's first image after the install, with the screen being recorded).

Measured

	Pixel 8a	Galaxy S26
Runtime	LiteRT 2.2.0 (Kotlin), every graph on Accelerator.GPU at FP32, Android 16	LiteRT 2.2.0 (Kotlin), every graph on Accelerator.GPU at FP32, Android 16
Code	the android/ engine in the iris-3b sample app: its Generate path, started by the debug verify extra (reference noise)	the android/ engine in the iris-3b sample app: its Generate path, started by the debug verify extra (reference noise)
First image after the install (apple prompt)	468.8 s, thermal status 0 at the start	212.6 s, thermal status 0 at the start
Cubes / lighthouse prompts	503.7 s / 487.5 s, each the first image after a fresh install, thermal status 0 at the start	280.9 s / 320.1 s, the second and third images in a row, thermal status 2 at the start of each
First image, where the time goes (load = GPU compile)	text encoder 38.0 s load + 5.6 s run; adapter 5.1 + 2.0 s; transformer 267.1 + 141.5 s; closing graphs 7.8 s	text encoder 27.2 + 2.2 s; adapter 5.6 + 0.9 s; transformer 100.1 + 74.4 s; closing graphs 1.6 s
Peak memory of the app (VmHWM)	5.0 GiB	4.6 GiB
Free memory (MemAvailable): before an image / lowest during one	3.6–3.8 GiB / 288–444 MiB	6.4–7.6 GiB / 3.6–3.8 GiB
Other processes closed by Android's low-memory killer during one image	116–169, never the app	2–67, never the app
Image vs PyTorch fp32, PSNR (apple / cubes / lighthouse)	106.76 / 70.92 / 73.09 dB	106.84 / 74.02 / 72.57 dB
Image vs PyTorch fp32, SSIM	1.0 / 1.0 / 1.0	1.0 / 1.0 / 0.99999
Measured	2026-10-21	2026-10-21

Prompts: "a red apple on a wooden table, studio lighting", "three blue glass cubes, two red spheres & one green cone on a white table; 45° top light, 50mm f/1.8 (sharp focus!)" and "夕暮れの海辺に立つ白い灯台、　柔らかな光と長い影" (a white lighthouse by the sea at dusk, soft light and long shadows). Reference: the upstream PyTorch pipeline with the text encoder and the transformer in fp32, from the same starting noise; PSNR and SSIM over the 768×768 image.

The int8 transformer setting on the Galaxy S26

iris3b_config_dit_int8_fp16.json runs the six trunk graphs with int8 weights and, with the pixel stage, at the GPU's default precision (fp16 arithmetic); the 12 text encoder graphs and the adapter stay at FP32. The two settings took turns on the Galaxy S26 in one session (FP32, int8, int8, FP32; the apple prompt from the upstream reference run's starting noise; each image a new process of the sample app; measured 2026-10-31):

	FP32	int8	int8	FP32
Image (apple prompt)	201.2 s	132.6 s	128.3 s	264.0 s
Thermal status at the start	0	2	2	2
Text encoder load + run	27.2 + 2.2 s	25.3 + 2.3 s	25.6 + 2.4 s	25.5 + 2.3 s
Adapter load + run	4.7 + 1.0 s	4.2 + 0.9 s	4.2 + 0.9 s	4.0 + 0.9 s
Transformer load + run (4 steps)	88.2 + 75.7 s	81.8 + 16.3 s	77.2 + 16.3 s	148.7 + 79.5 s
One step's run (median of 4)	18.8 s	4.1 s	4.1 s	20.0 s
Peak memory of the app (VmHWM)	4.62 GiB	3.19 GiB	3.19 GiB	4.67 GiB

These four images came through the app's benchmark screen (BenchActivity), started over adb; it runs the same engine code as the Generate button, so a Generate on the same phone and files gives the same image (checked on the apple prompt with the int8 setting: the Generate path's image is identical bit for bit). The first started on a rested phone; each of the other three started 32 s after the one before had finished, with Android's thermal status at 2 (0 = cool) and the CPU clock already capped, so the two int8 images and the second FP32 image ran on a hot phone and the first FP32 image on a rested one. The 201.2 s FP32 image here is the same kind of run as the 212.6 s in the table above (a rested phone); the 264.0 s was the fourth image in a row on a hot phone, so 232.6 s is the same-session comparison figure, not a first-image time.

What the int8 setting changes in the image: 12 prompts with 40 yes/no questions about what each prompt asks for (objects, colours, counts, written words, a position), each answered by Qwen3-VL-4B-Instruct looking at the image. The Galaxy S26's images with the int8 setting pass 39 of the 40, the same as the pictures the FP32 setting draws (made on a Mac GPU at FP32 from the same starting noise; the one miss in both: "exactly four apples"), and none came out blank or as pure noise. Those 12 images came from an earlier build of the same engine in a test app; on the apple prompt it and this sample app give the same image bit for bit, in both settings. The pictures themselves change: for the same seed the two settings draw different pictures, close for some prompts (PSNR 27.6 dB, the apple) and far apart for others (12.8 dB, the neon sign), while the objects, colours, counts and words the prompts asked for were still there (39 of 40 checks, the same as FP32). The Pixel 8a was not measured with the int8 setting; its numbers above are the FP32 setting's.

Settings measured but not shipped

iris3b_config.json ships "residency": "P1" (each graph is loaded for its run) with all 20 graphs on the GPU at FP32; every number in the first table above uses it. Two other settings were measured on the Galaxy S26:

• Text encoder on the CPU. In an earlier build of the same engine in a test app, both runs from thermal status 1: with the 12 text encoder graphs on the CPU (XNNPACK, 4 threads) and the rest on the GPU, the first image took 186.2 s, 7.6 % less than the 201.6 s with every graph on the GPU.
• P4: compile the next graph while the current one runs. The engine has this mode. Through the app's BenchActivity, it made the first image 149.1 s (from thermal status 0) and the second 211.1 s. During the third image in a row, while a compile overlapped a run, the GPU driver ended the app: Exiting the process com.iris3b from function cl_a8x_cmdbuf_mgr_submit_ibs.

Files

File (in model/)	What it is	Size
iris_te_c00_f16w.tflite … iris_te_c11_f16w.tflite	text encoder: Qwen3-VL-4B decoder layers 1–2, then 3 layers per graph up to layer 35; each graph's output is one of the 12 hidden-state taps	12 files, 7.07 GB
iris_dit_adapter_f16w_dit.tflite	text adapter: the 12 taps → 300 conditioning tokens (once per prompt)	0.50 GB
iris_dit_dual{0,1,2}_f16w_dit.tflite	dual-stream blocks 1–3, 4–6, 7–8 (dual0 embeds the image patches)	2.42 GB
iris_dit_single{0,1,2}_f16w_dit.tflite	single-stream blocks 1–6, 7–12, 13–16	2.41 GB
iris_dit_pixel_f16w_dit.tflite	pixel stage: the velocity of every pixel	0.47 GB
iris_dit_dual{0,1,2}_i8w_dit.tflite	the int8 setting's dual-stream graphs: the same blocks with int8 weights	1.21 GB
iris_dit_single{0,1,2}_i8w_dit.tflite	the int8 setting's single-stream graphs	1.21 GB
iris_embed_bf16.f32	token embedding table, raw little-endian bf16 [151936, 2560]	0.78 GB
iris_host.f32	host tensors (float32): rotary tables, the timestep embeddings and modulation rows of the 4 fixed steps, the step sigmas, the reference starting noise	58.7 MB
iris_tokenizer.json	tokenizer tables (vocab, merges, added tokens) and the chat-template ids	6.6 MB
iris3b_config.json	every graph's file, inputs and outputs, GPU precision FP32; host tensor offsets	19 kB
iris3b_config_dit_int8_fp16.json	the int8 setting: the same config with the six _i8w_ trunk graphs, and the trunk and the pixel stage at the GPU's default precision	19 kB
SHA256SUMS	sha256 of the files above

How it was converted

• litert-torch 0.9.4 exported the upstream PyTorch modules (iris-3b Turbo at revision adc05b9, Qwen3-VL-4B-Instruct at ebb281e) as fp32 graphs. ai-edge-quantizer 0.9.0 then stored the fully-connected weights as fp16 with litert-torch's full_fp16_recipe (explicit DEQUANTIZE, fp32 compute). The checkpoint's weights are bf16, which fp16 holds exactly except for values smaller than 2^-17 in magnitude, which are rounded; the adapter's attention-query rows, which are not bf16 values in the exported fp32 graph, are rounded too.
• The int8 setting's six trunk graphs were made from the _f16w_ graphs: each fp16 weight behind a DEQUANTIZE was turned back into an fp32 constant, then ai-edge-quantizer 0.9.0 applied litert-torch's dynamic int8 recipe (full_dynamic_recipe(INT8, CHANNELWISE): int8 weights, one scale per output channel).
• Turbo's 4 step sigmas are fixed, so the timestep embeddings and modulation rows of those steps are precomputed into iris_host.f32, with the rotary tables.
• The tokenizer is ported, not called: the Kotlin port and the Python port both give the Hugging Face tokenizer's ids on 15 test captions and 2,000 random strings.

Limits

• 768×768 only. Turbo was distilled at about one megapixel; in the upstream PyTorch pipeline, 512×512 and 256×256 come out broken.
• Prompts up to 295 tokens. After the chat template's 34-token prefix, the text encoder reads a 300-token window, and the template's suffix takes 5 of it; a longer prompt is cut (the sample app says so).
• Two GPU settings. iris3b_config.json sets FP32 for every graph and draws the pictures the quality numbers above refer to. The int8 setting is faster on the Galaxy S26 and answers the 40 prompt questions like FP32, but its pictures differ from the FP32 ones (PSNR 12.8–27.6 dB); it was measured on the Galaxy S26 only.
• No NPU path. On the Galaxy S26's NPU (Qualcomm HTP), the float versions (fp32 weights) of the first text-encoder graph and of the pixel stage, a one-layer text-encoder probe and a one-block transformer probe were tried. Each went onto the NPU whole (one partition), but the NPU computes in fp16, which does not hold the image: the transformer block's output had non-finite values.
• Storage and memory. The files take 13.7 GB on the phone (11.3 GB with the int8 setting). On the Pixel 8a, free memory was 3.6–3.8 GiB before an image and 288–444 MiB at its lowest during one, and Android closed 116–169 other processes during each image to make room. Close other apps first. The sample app reads free memory every 200 ms and stops before the next graph below 200,000 kB (about 195 MiB).
• Only the Turbo text-to-image checkpoint is converted (the upstream repository also has a base model, a depth model and an upscaler).

License

Apache-2.0 (LICENSE). The graphs, the embedding table, the host tensors and the tokenizer tables are derived from two Apache-2.0 works, listed with the changes in NOTICE:

• iris-3b — Copyright 2026 Speridlabs (model, code).
• Qwen3-VL-4B-Instruct — Alibaba Cloud (model): the text encoder layers, the token embedding table and the tokenizer.