# Non shuffle training

**URL:** <https://discuss.huggingface.co/t/non-shuffle-training/6986>\
**Category:** Beginners\
**Created:** [June 18, 2021, 9:37pm UTC](https://discuss.huggingface.co/t/non-shuffle-training/6986 "2021-06-18T21:37:43Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sarvghotra](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sarvghotra/32/1824_2.png) [@sarvghotra](https://discuss.huggingface.co/u/sarvghotra)\
**Post date:** [June 18, 2021, 9:37pm UTC](https://discuss.huggingface.co/t/non-shuffle-training/6986/1 "2021-06-18T21:37:43Z")

</div>

Hi there,

In order to debug something I need to make data non-shuffle. Can you please tell me how to turn off the shuffle?

I am using `from transformers import Trainer` for training and `from datasets import load_dataset` for data loading with default arguments.

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [June 21, 2021, 1:08pm UTC](https://discuss.huggingface.co/t/non-shuffle-training/6986/2 "2021-06-21T13:08:34Z")

</div>

There is no option to do this natively in the Trainer, you can either make a source install and change the line that creates the training dataloader, or subclass `Trainer` and override the `get_train_dataloader` method.

---

<div class="post-metadata">

**Author:** ![JackBAI](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jackbai/32/17827_2.png) [@JackBAI](https://discuss.huggingface.co/u/JackBAI)\
**Post date:** [September 3, 2023, 5:17am UTC](https://discuss.huggingface.co/t/non-shuffle-training/6986/3 "2023-09-03T05:17:36Z")

</div>

Refer to thread here: [How to ensure the dataset is shuffled for each epoch using Trainer and Datasets? - #3 by lhoestq](https://discuss.huggingface.co/t/how-to-ensure-the-dataset-is-shuffled-for-each-epoch-using-trainer-and-datasets/4212/3)

---

<div class="post-metadata">

**Author:** ![JackBAI](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jackbai/32/17827_2.png) [@JackBAI](https://discuss.huggingface.co/u/JackBAI)\
**Post date:** [September 3, 2023, 5:19am UTC](https://discuss.huggingface.co/t/non-shuffle-training/6986/4 "2023-09-03T05:19:02Z")

</div>

As non-shuffling is a minor demand from users, the Trainer class doesn’t provide this option to avoid careless users making mistakes.

---

<div class="post-metadata">

**Author:** ![Florient](https://avatars.discourse-cdn.com/v4/letter/f/cab0a1/32.png) [@Florient](https://discuss.huggingface.co/u/Florient)\
**Post date:** [July 5, 2024, 4:35am UTC](https://discuss.huggingface.co/t/non-shuffle-training/6986/5 "2024-07-05T04:35:02Z")

</div>

where should i change to make it non-shuffled？

---

<div class="post-metadata">

**Author:** ![fedyanin](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/fedyanin/32/30885_2.png) [@fedyanin](https://discuss.huggingface.co/u/fedyanin)\
**Post date:** [August 26, 2024, 9:31am UTC](https://discuss.huggingface.co/t/non-shuffle-training/6986/6 "2024-08-26T09:31:01Z")

</div>

For the reference, the way to go was to switch train sampler. E.g. this worked for me for SFTTrainer, should be the same for vanilla Trainer. You inherit from initial class and then use your new redefined one.

```auto
from torch.utils.data import SequentialSampler
from trl import SFTTrainer

class SFTTrainer2(SFTTrainer):
    def _get_train_sampler(self):
        return SequentialSampler(self.train_dataset)

```

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [August 26, 2024, 1:30pm UTC](https://discuss.huggingface.co/t/non-shuffle-training/6986/7 "2024-08-26T13:30:38Z")

</div>

Hi,

It would look like so:

```python
from transformers import Trainer
from torch.utils.data import DataLoader

class MyCustomTrainer(Trainer):
    def get_train_dataloader(self) -> DataLoader:
       train_dataset = self.train_dataset
       return DataLoader(train_dataset, shuffle=False, batch_size=...)

```

Note that this is just a minimal version for demo purposes, refer to the [source code](https://github.com/huggingface/transformers/blob/19e6e80e10118f855137b90740936c0b11ac397f/src/transformers/trainer.py#L897) if you want to tweak it more.