# Missing keys "model.embeddings.position\_ids" when loading model using state\_dict

**URL:** <https://discuss.huggingface.co/t/missing-keys-model-embeddings-position-ids-when-loading-model-using-state-dict/744>\
**Category:** Beginners\
**Created:** [August 18, 2020, 7:58am UTC](https://discuss.huggingface.co/t/missing-keys-model-embeddings-position-ids-when-loading-model-using-state-dict/744 "2020-08-18T07:58:24Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jyliu](https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/32.png) [@jyliu](https://discuss.huggingface.co/u/jyliu)\
**Post date:** [August 18, 2020, 7:58am UTC](https://discuss.huggingface.co/t/missing-keys-model-embeddings-position-ids-when-loading-model-using-state-dict/744/1 "2020-08-18T07:58:24Z")

</div>

I have saved the model like

> model\_state\_dict = model.module.state\_dict()  
> torch.save({‘model\_state\_dict’: model\_state\_dict}, osp.join(save\_dir, ‘best.ckpt’))

Now I try to load the model like

> model\_path = “./models/best.ckpt”  
> ckpt = torch.load(model\_path)  
> model.load\_state\_dict(ckpt[‘model\_state\_dict’])

then it has the error. There are “position\_ids” in model, but not in the saved ckpt file.  
Anyway to skip loading position\_ids ?

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [August 19, 2020, 5:29pm UTC](https://discuss.huggingface.co/t/missing-keys-model-embeddings-position-ids-when-loading-model-using-state-dict/744/2 "2020-08-19T17:29:39Z")

</div>

Hi @jyliu, is there any specif reason for not using `.save_pretrained` and `.from_pretrained`

---

<div class="post-metadata">

**Author:** ![jyliu](https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/32.png) [@jyliu](https://discuss.huggingface.co/u/jyliu)\
**Post date:** [August 20, 2020, 3:47am UTC](https://discuss.huggingface.co/t/missing-keys-model-embeddings-position-ids-when-loading-model-using-state-dict/744/3 "2020-08-20T03:47:43Z")

</div>

Thanks for your reply. Because the bert-model is a part of my whole model. I directly saved the whole model. So what is the best practice under this situation ?

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [August 20, 2020, 2:39pm UTC](https://discuss.huggingface.co/t/missing-keys-model-embeddings-position-ids-when-loading-model-using-state-dict/744/4 "2020-08-20T14:39:18Z")

</div>

Will it be possible for you to create colab for this ?

Also, to take advantage of `.from_pretrained` and `.save_pretrained` you can sub-class the `BertPretrainedModel` and add the additional layers in it. See [these](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L1267) task specific bert models, they use bert and additional layer on top of it and subclass `BertPreTrainedModel`.

let me know if this solves your problem.

---

<div class="post-metadata">

**Author:** ![jyliu](https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/32.png) [@jyliu](https://discuss.huggingface.co/u/jyliu)\
**Post date:** [August 21, 2020, 4:02am UTC](https://discuss.huggingface.co/t/missing-keys-model-embeddings-position-ids-when-loading-model-using-state-dict/744/5 "2020-08-21T04:02:42Z")

</div>

Thanks. This will work for me.