# How to train TFT5ForConditionalGeneration model?

**URL:** <https://discuss.huggingface.co/t/how-to-train-tft5forconditionalgeneration-model/888>\
**Category:** 🤗Transformers\
**Created:** [August 27, 2020, 9:18am UTC](https://discuss.huggingface.co/t/how-to-train-tft5forconditionalgeneration-model/888 "2020-08-27T09:18:08Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![brand17](https://avatars.discourse-cdn.com/v4/letter/b/5f8ce5/32.png) [@brand17](https://discuss.huggingface.co/u/brand17)\
**Post date:** [August 27, 2020, 9:18am UTC](https://discuss.huggingface.co/t/how-to-train-tft5forconditionalgeneration-model/888/1 "2020-08-27T09:18:08Z")

</div>

My code is as follows:

```auto
batch_size=8
sequence_length=25
vocab_size=100
import tensorflow as tf
from transformers import T5Config, TFT5ForConditionalGeneration
configT5 = T5Config(
    vocab_size=vocab_size,
    d_ff =512, 
)  
model = TFT5ForConditionalGeneration(configT5)

model.compile(
    optimizer = tf.keras.optimizers.Adam(),
    loss = tf.keras.losses.SparseCategoricalCrossentropy()
)
input = tf.random.uniform([batch_size,sequence_length],0,vocab_size,dtype=tf.int32)
labels = tf.random.uniform([batch_size,sequence_length],0,vocab_size,dtype=tf.int32)
input = {'inputs': input, 'decoder_input_ids': input}
model.fit(input, labels)

```

It generates an error:

> logits and labels must have the same first dimension, got logits shape [1600,64] and labels shape [200] [[node sparse\_categorical\_crossentropy\_3/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits (defined at C:\Users\FA.PROJECTOR-MSK\Google Диск\Colab Notebooks\PoetryTransformer\experiments\TFT5.py:30)]] [Op:\_\_inference\_train\_function\_25173] Function call stack: train\_function

I dont understand - why the model returns a tensor of [1600, 64]. According to [T5](https://huggingface.co/transformers/model_doc/t5.html#tft5forconditionalgeneration) model returns [batch\_size, sequence\_len, vocab\_size].

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [August 28, 2020, 3:32pm UTC](https://discuss.huggingface.co/t/how-to-train-tft5forconditionalgeneration-model/888/2 "2020-08-28T15:32:29Z")

</div>

Pinging @patrickvonplaten

---

<div class="post-metadata">

**Author:** ![patrickvonplaten](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/patrickvonplaten/32/2171_2.png) [@patrickvonplaten](https://discuss.huggingface.co/u/patrickvonplaten)\
**Post date:** [September 1, 2020, 10:26am UTC](https://discuss.huggingface.co/t/how-to-train-tft5forconditionalgeneration-model/888/3 "2020-09-01T10:26:47Z")

</div>

Thanks for posting this! I’ll open an issue about it on work on it asap.  
Issue: [https://github.com/huggingface/transformers/issues/6876](https://github.com/huggingface/transformers/issues/6876)

---

<div class="post-metadata">

**Author:** ![congcongwang](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/congcongwang/32/3282_2.png) [@congcongwang](https://discuss.huggingface.co/u/congcongwang)\
**Post date:** [September 6, 2020, 12:02pm UTC](https://discuss.huggingface.co/t/how-to-train-tft5forconditionalgeneration-model/888/4 "2020-09-06T12:02:58Z")

</div>

Using high-level keras to fine-tune T5 seems tricky due to limited flexibility and code controls. I wrote something on fine-tuning T5 with customized training loop in TF2.0: [https://github.com/wangcongcong123/ttt](https://github.com/wangcongcong123/ttt). Hope this helps.

---

<div class="post-metadata">

**Author:** ![patrickvonplaten](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/patrickvonplaten/32/2171_2.png) [@patrickvonplaten](https://discuss.huggingface.co/u/patrickvonplaten)\
**Post date:** [October 1, 2020, 3:56pm UTC](https://discuss.huggingface.co/t/how-to-train-tft5forconditionalgeneration-model/888/5 "2020-10-01T15:56:26Z")

</div>

This should help 🙂 [https://github.com/huggingface/transformers/pull/7428](https://github.com/huggingface/transformers/pull/7428)

---

<div class="post-metadata">

**Author:** ![brand17](https://avatars.discourse-cdn.com/v4/letter/b/5f8ce5/32.png) [@brand17](https://discuss.huggingface.co/u/brand17)\
**Post date:** [November 21, 2020, 10:19am UTC](https://discuss.huggingface.co/t/how-to-train-tft5forconditionalgeneration-model/888/6 "2020-11-21T10:19:00Z")

</div>

Thanks a lot. Am I right that I have to override train\_step() to make TFT5ForConditionalGeneration working ?