M3OPD-Reasoner: An Open Recipe for Post-Training Multimodal Reasoning with Multi-Teacher On-Policy Distillation
Abstract
Multi-Teacher On-Policy Distillation (MOPD) enables a single student to absorb the skills of several domain teachers by learning from its own generated responses. We present Multimodal Model Multi-Teacher On-Policy Distillation Reasoner (M3OPD-Reasoner), a fully open recipe for post-training vision-language models (VLMs) over three reasoning domains: image STEM, image perception, and video. We curate two datasets, M3OPD-SFT-1.35M for supervised fine-tuning (SFT) and M3OPD-RL-193K for reinforcement learning (RL) with verifiable rewards. On these data, we compare post-training recipes that start from Qwen3.5-9B: training one model on the mixed data of all domains (MixSFT and MixRL), and training one teacher per domain with Group Sequence Policy Optimization (GSPO), directly or after an SFT warm-up, and distilling the teachers into one student with MOPD. We evaluate on 21 image STEM, image perception, and video reasoning benchmarks after we decontaminate the training data against them. The comparison establishes MOPD of teachers trained with GSPO directly from Qwen3.5-9B as the best recipe: the student closely matches each teacher on its own domain and is on average above the mixed-training baselines on the same prompts. Since MOPD already merges the domain capabilities that MixRL would otherwise have to learn, its student is a stronger starting point for RL, and we ask whether the same data can carry the model further from there: we add MixRL on top of the MOPD student, on the same prompts and with the same settings, which, to our knowledge, no prior VLM recipe has studied. The same prompts still improve the MOPD student, with no collapse, and give our main model, M3OPD-Reasoner-9B. MOPD and MixRL can therefore work as consecutive stages that get more out of the same data. We release all data, the training code, and the checkpoints of every stage.
Community
M3OPD-Reasoner is a fully open recipe that post-trains Qwen3.5-9B into one reasoning model for image STEM, image perception and video.
š Project page: https://ngquangtrung57.github.io/m3opd-reasoner-page/
š» Code: https://github.com/ngquangtrung57/M3OPD-Reasoner
š¤ Models (all stages): https://huggingface.co/collections/ngqtrung/m3opd-reasoner-6aed5164d151b6643a1b3b2d
š SFT data: https://huggingface.co/datasets/ngqtrung/M3OPD-SFT-1.35M
š RL data: https://huggingface.co/datasets/ngqtrung/M3OPD-RL-193K
š¬ alphaXiv: https://www.alphaxiv.org/abs/2611.23034
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Replay for LLM Continual Learning with Multi-Teacher On-Policy Distillation (2026)
- MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoning (2026)
- Rethinking Self-Distillation for Multi-Teacher Capability Merging (2026)
- Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation (2026)
- Latent-MOPD: Latent Multi-Teacher On-Policy Distillation (2026)
- GRASP: Group-Adaptive Hybrid RL-SFT Post-Training for Medical VQA without Reasoning Annotations (2026)
- Understanding the Synergy between SFT, RLVR, and OPD in LLM Post-Training (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2611.23034 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 16
ngqtrung/M3OPD-Warmup-SFT-Video-9B
Datasets citing this paper 2
ngqtrung/M3OPD-SFT-1.35M
ngqtrung/M3OPD-RL-193K
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper