# Unbelievable Error: Help ME!

**URL:** <https://discuss.huggingface.co/t/unbelievable-error-help-me/129971>\
**Category:** 🤗Transformers\
**Created:** [December 8, 2024, 12:11pm UTC](https://discuss.huggingface.co/t/unbelievable-error-help-me/129971 "2024-12-08T12:11:11Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![richardlevinson3](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/richardlevinson3/32/37833_2.png) [@richardlevinson3](https://discuss.huggingface.co/u/richardlevinson3)\
**Post date:** [December 8, 2024, 12:11pm UTC](https://discuss.huggingface.co/t/unbelievable-error-help-me/129971/1 "2024-12-08T12:11:11Z")

</div>

Hey Guys, I got unbelievable Error.

ValueError: You have set `args.eval_strategy` to steps but you didn’t pass an `eval_dataset` to `Trainer`. Either set `args.eval_strategy` to `no` or pass an `eval_dataset`.

My training code is:

```auto
training_args = TrainingArguments(
    output_dir="/kaggle/working/twitter-sentiment-analysis-llm",
    report_to="wandb",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=3,
    weight_decay=0.01,
    eval_strategy="steps",
    save_strategy="steps",
    load_best_model_at_end=True,
    eval_steps=500,
    save_steps=500,
    logging_steps=10,
    fp16=True,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_ds['train'],
    eval_dataset=train_ds['test'],
    tokenizer=tokenizer,
    data_collator=data_collator,
    compute_metrics=compute_metrics,
)

trainer.train()

```

I checked eval\_dataset and passed it exactly. But I got an error after all training steps.

Guys, Please Help me!!

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [December 8, 2024, 12:38pm UTC](https://discuss.huggingface.co/t/unbelievable-error-help-me/129971/2 "2024-12-08T12:38:18Z")

</div>

There seem to be several patterns… I hope it’s not a new bug.

> [@ValueError: Trainer: evaluation requires an eval\_dataset](https://discuss.huggingface.co/t/valueerror-trainer-evaluation-requires-an-eval-dataset/40670):
>
> I’m trying to do a finetuning without an evaluation dataset. For that, I’m using the following code: training\_args = TrainingArguments( output\_dir=resume\_from\_checkpoint, evaluation\_strategy="epoch", per\_device\_train\_batch\_size=1, ) def compute\_metrics(pred: EvalPrediction): labels = pred.label\_ids preds = pred.predictions.argmax(-1) f1 = f1\_score(labels, preds, average="weighted") acc = accuracy\_score(labels, preds, average="…

> You can avoid this error by not specifying `evaluation_strategy` in `TrainingArguments`.

> <https://stackoverflow.com/questions/76310533/how-to-fix-trainer-evaluation-requires-an-eval-dataset-in-huggingface-transfo>

---

<div class="post-metadata">

**Author:** ![richardlevinson3](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/richardlevinson3/32/37833_2.png) [@richardlevinson3](https://discuss.huggingface.co/u/richardlevinson3)\
**Post date:** [December 9, 2024, 2:03am UTC](https://discuss.huggingface.co/t/unbelievable-error-help-me/129971/3 "2024-12-09T02:03:44Z")

</div>

Thanks for your comment.  
You’re right, but I 'd like to use evaluation step.

---

<div class="post-metadata">

**Author:** ![ipzk201-kio](https://avatars.discourse-cdn.com/v4/letter/i/5daacb/32.png) [@ipzk201-kio](https://discuss.huggingface.co/u/ipzk201-kio)\
**Post date:** [December 11, 2024, 4:54pm UTC](https://discuss.huggingface.co/t/unbelievable-error-help-me/129971/4 "2024-12-11T16:54:24Z")

</div>

+1

Any workarounds yet?

UDP: In my case it was a problem with dataset separation. Try len() both of your train and validation dataset

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [December 12, 2024, 12:53am UTC](https://discuss.huggingface.co/t/unbelievable-error-help-me/129971/5 "2024-12-12T00:53:35Z")

</div>

Should we raise an issue on the github for the datasets library?😅  
It’s not clear whether it’s a bug or not, but at the very least the error message is not appropriate.

> **[Issues · huggingface/datasets](https://github.com/huggingface/datasets/issues)**
>
> 🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools - Issues · huggingface/datasets

---

<div class="post-metadata">

**Author:** ![mettinski](https://avatars.discourse-cdn.com/v4/letter/m/a88e4f/32.png) [@mettinski](https://discuss.huggingface.co/u/mettinski)\
**Post date:** [December 18, 2024, 12:06pm UTC](https://discuss.huggingface.co/t/unbelievable-error-help-me/129971/6 "2024-12-18T12:06:51Z")

</div>

Had the same issue, was mentioned [here](https://github.com/huggingface/transformers/issues/34701) and fixed [here](https://github.com/huggingface/transformers/commit/7b3d615bc23c40daa9f38dbae0abd979936f6414). `pip install transformers==4.47.1` fixed it for me

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [December 19, 2024, 9:06am UTC](https://discuss.huggingface.co/t/unbelievable-error-help-me/129971/7 "2024-12-19T09:06:53Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.