# Does starting training from a previous checkpoint reset the learning rate?

**URL:** <https://discuss.huggingface.co/t/does-starting-training-from-a-previous-checkpoint-reset-the-learning-rate/9247>\
**Category:** 🤗Transformers\
**Created:** [August 14, 2021, 10:02pm UTC](https://discuss.huggingface.co/t/does-starting-training-from-a-previous-checkpoint-reset-the-learning-rate/9247 "2021-08-14T22:02:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![md1630](https://avatars.discourse-cdn.com/v4/letter/m/977dab/32.png) [@md1630](https://discuss.huggingface.co/u/md1630)\
**Post date:** [August 14, 2021, 10:02pm UTC](https://discuss.huggingface.co/t/does-starting-training-from-a-previous-checkpoint-reset-the-learning-rate/9247/1 "2021-08-14T22:02:29Z")

</div>

Hi,

I want to start training a new model by loading a previous model I trained, I want to know what happens to the learning rate in this case – does it start at the learning rate I set, or does it start from the prev learning rate of the checkpoint?

---

<div class="post-metadata">

**Author:** ![ugaray96](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ugaray96/32/12082_2.png) [@ugaray96](https://discuss.huggingface.co/u/ugaray96)\
**Post date:** [January 20, 2022, 10:13am UTC](https://discuss.huggingface.co/t/does-starting-training-from-a-previous-checkpoint-reset-the-learning-rate/9247/2 "2022-01-20T10:13:30Z")

</div>

Hi!

As far as I have experienced, it continues from the last learning rate number saved in the checkpoint from where you resumed.  
It also starts from the last epoch number.

---

<div class="post-metadata">

**Author:** ![jbmaxwell](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@jbmaxwell](https://discuss.huggingface.co/u/jbmaxwell)\
**Post date:** [August 20, 2022, 11:17pm UTC](https://discuss.huggingface.co/t/does-starting-training-from-a-previous-checkpoint-reset-the-learning-rate/9247/3 "2022-08-20T23:17:50Z")

</div>

I’m sure you’ve long since realized this (or maybe never had the problem), but I thought I’d mention it for the benefit of anyone else with questions about resuming.

I embarrassingly failed to give the checkpoint to **both** my model **and** my trainer, and only this morning realized that it won’t resume correctly this way. At the very least it will fail to continue from the correct training epoch and step, but I think it also impacts the scheduler in more subtle ways, since I got significantly poorer training results when loading the checkpoint _only to the model_, even though I manually set the learning rate (i.e., to the one stored in the checkpoint). Not sure exactly why this is… (non-linear learning rate decay, maybe?)

Anyway, hopefully this helps someone somewhere sometime. 🙂