# GPT2 finetuned with eos token will never yield eos token during generation **URL:** \ **Category:** Beginners\ **Created:** [March 7, 2022, 10:30am UTC](https://discuss.huggingface.co/t/gpt2-finetuned-with-eos-token-will-never-yield-eos-token-during-generation/15437 "2022-03-07T10:30:16Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![darkraipro](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/darkraipro/32/4196_2.png) [@darkraipro](https://discuss.huggingface.co/u/darkraipro)\ **Post date:** [March 7, 2022, 10:30am UTC](https://discuss.huggingface.co/t/gpt2-finetuned-with-eos-token-will-never-yield-eos-token-during-generation/15437/1 "2022-03-07T10:30:16Z")
Hi, I finetune the smallest version of gpt2 (distilgpt2) trained on a dataset. The dataset consists only of texts and after some texts, an EOS token is inserted. Training is running decently, the loss is constantly decreasing. But using model.generate(input\_ids, …) no matter what the model will always output tokens till the max\_length has been reached. I think that the probability for EOS token has not been adjusted in the model well enough. Any tips to improve it or make the model generate EOS after some texts? ---
**Author:** ![zhensuuu](https://avatars.discourse-cdn.com/v4/letter/z/6f9a4e/32.png) [@zhensuuu](https://discuss.huggingface.co/u/zhensuuu)\ **Post date:** [May 31, 2022, 7:14am UTC](https://discuss.huggingface.co/t/gpt2-finetuned-with-eos-token-will-never-yield-eos-token-during-generation/15437/2 "2022-05-31T07:14:09Z")
I have encountered the same problem. Have you found any solutions? ---
**Author:** ![zhensuuu](https://avatars.discourse-cdn.com/v4/letter/z/6f9a4e/32.png) [@zhensuuu](https://discuss.huggingface.co/u/zhensuuu)\ **Post date:** [June 2, 2022, 1:05pm UTC](https://discuss.huggingface.co/t/gpt2-finetuned-with-eos-token-will-never-yield-eos-token-during-generation/15437/3 "2022-06-02T13:05:56Z")
Finally, I found the reason. The DataCollatorForLanguageModeling always masks the pad\_token in the labels and I set the pad\_token = eos\_token. ---
**Author:** ![larsdw](https://avatars.discourse-cdn.com/v4/letter/l/7c8e57/32.png) [@larsdw](https://discuss.huggingface.co/u/larsdw)\ **Post date:** [May 26, 2023, 9:17am UTC](https://discuss.huggingface.co/t/gpt2-finetuned-with-eos-token-will-never-yield-eos-token-during-generation/15437/4 "2023-05-26T09:17:44Z")
Hi, I am encountering the same problem. How did you resolve this? this you change the pad\_token to something else? ---
**Author:** ![JoaoLages](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@JoaoLages](https://discuss.huggingface.co/u/JoaoLages)\ **Post date:** [May 28, 2023, 1:37am UTC](https://discuss.huggingface.co/t/gpt2-finetuned-with-eos-token-will-never-yield-eos-token-during-generation/15437/5 "2023-05-28T01:37:24Z")
I have the same problem, the model does not shut up… ---
**Author:** ![amilios](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/amilios/32/17759_2.png) [@amilios](https://discuss.huggingface.co/u/amilios)\ **Post date:** [August 5, 2023, 2:15am UTC](https://discuss.huggingface.co/t/gpt2-finetuned-with-eos-token-will-never-yield-eos-token-during-generation/15437/6 "2023-08-05T02:15:38Z")
> **[Solving The Issue of Falcon Text Generation Never Stopping](https://medium.com/@mayvic/solving-the-issue-of-falcon-text-generation-never-stopping-e8f599eae8f0)** > > How to make an overly chatty bird stop talking. I believe the most elegant solution may be to switch to using the Seq2Seq DataCollator as described here, otherwise you can introduce a new padding token. ---
**Author:** ![Sam1989](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sam1989/32/25757_2.png) [@Sam1989](https://discuss.huggingface.co/u/Sam1989)\ **Post date:** [April 12, 2024, 9:10am UTC](https://discuss.huggingface.co/t/gpt2-finetuned-with-eos-token-will-never-yield-eos-token-during-generation/15437/7 "2024-04-12T09:10:12Z")
I faced the same problem. The fine-tuning of Gemma 2 works well according to the loss functions. But after training the prediction was just eos eos. The solution in my case was simple: Set eos\_token to False model = AutoModelForCausalLM.from\_pretrained(model\_id, tokenizer = AutoTokenizer.from\_pretrained(model\_id, add\_eos\_token=False)