# Loss Issues on Finetuning

**URL:** <https://discuss.huggingface.co/t/loss-issues-on-finetuning/74315>\
**Category:** Beginners\
**Created:** [February 22, 2024, 9:39am UTC](https://discuss.huggingface.co/t/loss-issues-on-finetuning/74315 "2024-02-22T09:39:52Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![zeynelulusan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/zeynelulusan/32/30652_2.png) [@zeynelulusan](https://discuss.huggingface.co/u/zeynelulusan)\
**Post date:** [February 22, 2024, 9:39am UTC](https://discuss.huggingface.co/t/loss-issues-on-finetuning/74315/1 "2024-02-22T09:39:52Z")

</div>

Hello,

Sometimes while finetuning an LLM, the train and validation loss decreases smoothly but the model does not learn the behavior properly. What could be the reasons for this? Could it be related to the decoding strategy? Or Dataset?

Thank you.