# Questions about loading checkpoint by \`.from\_pretrained\` **URL:** \ **Category:** 🤗Transformers\ **Created:** [September 29, 2025, 8:55pm UTC](https://discuss.huggingface.co/t/questions-about-loading-checkpoint-by-from-pretrained/168787 "2025-09-29T20:55:07Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![Zoe0427](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/zoe0427/32/53729_2.png) [@Zoe0427](https://discuss.huggingface.co/u/Zoe0427)\ **Post date:** [September 29, 2025, 8:55pm UTC](https://discuss.huggingface.co/t/questions-about-loading-checkpoint-by-from-pretrained/168787/1 "2025-09-29T20:55:07Z")
I tried to use BERT -large-uncased to finetune on GLUE bechmark, and I loaded checkpoint from Hub. When I saw the details: ![2685b2c07b56768f00bbb3b603f25612](https://us1.discourse-cdn.com/hellohellohello/original/3X/5/7/5726b3e3f3762befb6a7a7cc63879f45ac3f366b.png) My understand is that the finetuning model for GLUE is BERT + classifier, the chechpoint is only for BERT. So missing keys is belong to classifer. But I think there is a pooler behind BERT attention, so pooler keys are also missing, right? (But why I didn’t see them in missing keys?) And I think the unexpected keys actually can be assigned to pooler. And I also noticed that during the finetuning, not loading pooler’s weights (keeping the weights initialized normally) is better.🤔 ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [September 29, 2025, 11:54pm UTC](https://discuss.huggingface.co/t/questions-about-loading-checkpoint-by-from-pretrained/168787/2 "2025-09-29T23:54:40Z")
Seems [basically expected behavior](https://huggingface.co/datasets/John6666/forum1/blob/main/cp_loading_missing_keys.md).