# Explicitly set number of training steps using Trainer

**URL:** <https://discuss.huggingface.co/t/explicitly-set-number-of-training-steps-using-trainer/1127>\
**Category:** 🤗Transformers\
**Created:** [September 14, 2020, 7:11pm UTC](https://discuss.huggingface.co/t/explicitly-set-number-of-training-steps-using-trainer/1127 "2020-09-14T19:11:17Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![aclifton314](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aclifton314/32/55465_2.png) [@aclifton314](https://discuss.huggingface.co/u/aclifton314)\
**Post date:** [September 14, 2020, 7:11pm UTC](https://discuss.huggingface.co/t/explicitly-set-number-of-training-steps-using-trainer/1127/1 "2020-09-14T19:11:17Z")

</div>

I’m using `Trainer` to handle finetuning a GPT2 model. I see in `TrainingArguments` there is a `max_steps` that overrides `num_train_epochs`.

For a batch size of 32, is setting `max_steps=1000000` the equivalent of setting `num_train_epochs=31250`?

Also, what happens if I have a batch size of 6 but want to set `max_steps=1000000`? Does `Trainer` stop training after the nearest divisible whole number or does it change the batch size at the end?

Thanks in advance!!

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [September 14, 2020, 9:00pm UTC](https://discuss.huggingface.co/t/explicitly-set-number-of-training-steps-using-trainer/1127/2 "2020-09-14T21:00:14Z")

</div>

The number of steps is the number of update steps. It’s not the number of training examples to be seen.

---

<div class="post-metadata">

**Author:** ![aclifton314](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aclifton314/32/55465_2.png) [@aclifton314](https://discuss.huggingface.co/u/aclifton314)\
**Post date:** [September 14, 2020, 9:03pm UTC](https://discuss.huggingface.co/t/explicitly-set-number-of-training-steps-using-trainer/1127/3 "2020-09-14T21:03:51Z")

</div>

Ok. Is it then the case that for a batch size of 32, setting `max_steps=1000000` is the same as setting `num_train_epochs=31250` ?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [September 15, 2020, 12:01pm UTC](https://discuss.huggingface.co/t/explicitly-set-number-of-training-steps-using-trainer/1127/4 "2020-09-15T12:01:41Z")

</div>

No (unless your dataset has 32 batches so is of length 32\*32). `num_train_epochs = max_steps / len(train_dataloader)` in general.

---

<div class="post-metadata">

**Author:** ![aclifton314](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aclifton314/32/55465_2.png) [@aclifton314](https://discuss.huggingface.co/u/aclifton314)\
**Post date:** [September 15, 2020, 4:45pm UTC](https://discuss.huggingface.co/t/explicitly-set-number-of-training-steps-using-trainer/1127/5 "2020-09-15T16:45:05Z")

</div>

Apologies for my confusion. Is then `max_steps` the maximum number of times a batch makes a forward and backward pass through the network?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [September 16, 2020, 1:06am UTC](https://discuss.huggingface.co/t/explicitly-set-number-of-training-steps-using-trainer/1127/6 "2020-09-16T01:06:13Z")

</div>

Not exactly it is the number of update steps (so if you’re using gradient accumulation, it’s a bit different than just doing forward and backward).
