GATS Cold-Start: Pure-OPD cold-start (SciWorld)

Part of the Cold-start collection for the GATS paper (capability-aware on-policy distillation). Student: Qwen2.5-7B-Instruct; teacher: Qwen2.5-1.5B GRPO-trained on SciWorld train tasks.

Cold-start phase: pure on-policy distillation (OPD)

The student rolls out on-policy in the environment and imitates the Qwen2.5-1.5B teacher (GRPO step-150 on SciWorld train tasks) with a pure imitation objective (opd_only, coefficient 1.0 -- no RL reward term), for N steps where N is the GATS handoff step on this dataset: N = 29.

  • Rollouts: 16 tasks/step, group n=8; lr 1e-6; N = 29 steps
  • Checkpoint: global_step_29 (merged FSDP -> HF, bfloat16)
  • Validation before any RL: 18.0% (train-time val at RL step 0)

Intended use

Research on cold-start strategies for agent RL (imitation-first vs RL-from-scratch) and reproduction of the GATS paper baselines. Not a general-purpose assistant.

Provenance

Trained 2026-09-28 on the internal verl-based stack; checkpoints uploaded unmodified from the training runs (bfloat16 unless noted float32 above).

Downloads last month
19
Safetensors
Model size
8B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for C12H18/gats-coldstart-opd-qwen2.5-7b-sciworld

Base model

Qwen/Qwen2.5-7B
Finetuned
(3,275)
this model

Collection including C12H18/gats-coldstart-opd-qwen2.5-7b-sciworld