# Text Binary Classification with Byt5

**URL:** <https://discuss.huggingface.co/t/text-binary-classification-with-byt5/19366>\
**Category:** Models\
**Created:** [June 20, 2022, 7:48am UTC](https://discuss.huggingface.co/t/text-binary-classification-with-byt5/19366 "2022-06-20T07:48:13Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chandrai](https://avatars.discourse-cdn.com/v4/letter/c/8baadc/32.png) [@Chandrai](https://discuss.huggingface.co/u/Chandrai)\
**Post date:** [June 20, 2022, 7:48am UTC](https://discuss.huggingface.co/t/text-binary-classification-with-byt5/19366/1 "2022-06-20T07:48:13Z")

</div>

I am doing Text Classification with Byt5 model. I am pre-training with Simple Transformers library.  
Byt5 model always predicts only 1 class as output, giving very low accuracy, no matter how much ever hyper-parameter tuning I do. Same use-case with T5 or MT5 giving good results.

Have anyone else faced the issue with byt5 model?