# Arabic Question Generation using Shared AraBERT2AraBERT isn't working

**URL:** <https://discuss.huggingface.co/t/arabic-question-generation-using-shared-arabert2arabert-isnt-working/46928>\
**Category:** 🤗Transformers\
**Created:** [July 15, 2023, 4:50pm UTC](https://discuss.huggingface.co/t/arabic-question-generation-using-shared-arabert2arabert-isnt-working/46928 "2023-07-15T16:50:23Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lara15](https://avatars.discourse-cdn.com/v4/letter/l/dbc845/32.png) [@Lara15](https://discuss.huggingface.co/u/Lara15)\
**Post date:** [July 15, 2023, 4:50pm UTC](https://discuss.huggingface.co/t/arabic-question-generation-using-shared-arabert2arabert-isnt-working/46928/1 "2023-07-15T16:50:23Z")

</div>

I am working on Arabic question generation using arabert\_base, and mMarco dataset. I am following BERT2BERT for [CNN/Dailymail notebook](https://colab.research.google.com/drive/1Ekd5pUeCX7VOrMx94_czTkwNtLN32Uyu?usp=sharing#scrollTo=oOoSrwWarJAC), and the training notebook on [Arabic Empathetic Chatbot repo](https://github.com/aub-mind/Arabic-Empathetic-Chatbot/tree/master/model).

**The problem** is that (rouge, bleu, meteor) metrics all zeros, and the generated output is **[CLS] [CLS] [CLS] [CLS] [CLS] [CLS] [CLS] [CLS] [CLS]** repeated CLS token until the sentence reaches the maximum length. I am training on small subset to check if the model works fine before the full training.

I want to know if the small training set(10000 sample) is responsible for the problem, or the preprocessing method, or anything else.  
please check [my notebook](https://colab.research.google.com/drive/1fVVckmdh4EXWx6ag4cwVY02ESm-VNRNl?authuser=1#scrollTo=VDZOiAisaBVY)