# How to modify loss function in a seq2seq trainer?

**URL:** <https://discuss.huggingface.co/t/how-to-modify-loss-function-in-a-seq2seq-trainer/104680>\
**Category:** 🤗Transformers\
**Created:** [August 30, 2024, 3:15pm UTC](https://discuss.huggingface.co/t/how-to-modify-loss-function-in-a-seq2seq-trainer/104680 "2024-08-30T15:15:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![dmehta01](https://avatars.discourse-cdn.com/v4/letter/d/a6a055/32.png) [@dmehta01](https://discuss.huggingface.co/u/dmehta01)\
**Post date:** [August 30, 2024, 3:15pm UTC](https://discuss.huggingface.co/t/how-to-modify-loss-function-in-a-seq2seq-trainer/104680/1 "2024-08-30T15:15:16Z")

</div>

I am trying to fine tune a whisper model using this source: [Fine-Tune Whisper For Multilingual ASR with 🤗 Transformers](https://huggingface.co/blog/fine-tune-whisper)

I want to modify the loss function used to fine tune it. For example, I would like to modify the loss function to be able to distill knowledge from another ASR model.

So how do I modify the loss function and how would I do the knowledge distillation part as well?

@sanchit-gandhi Would appreciate it if you can help!

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [August 31, 2024, 8:36am UTC](https://discuss.huggingface.co/t/how-to-modify-loss-function-in-a-seq2seq-trainer/104680/2 "2024-08-31T08:36:57Z")

</div>

Hi,

You can overwrite the Seq2SeqTrainer’s compute\_loss method as shown [here](https://discuss.huggingface.co/t/specify-loss-for-trainer-trainingarguments/10481/2).