# BigBirdPegasus vs BigBird

**URL:** <https://discuss.huggingface.co/t/bigbirdpegasus-vs-bigbird/10340>\
**Category:** 🤗Transformers\
**Created:** [September 27, 2021, 5:07pm UTC](https://discuss.huggingface.co/t/bigbirdpegasus-vs-bigbird/10340 "2021-09-27T17:07:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![SajjadAyoubi](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sajjadayoubi/32/2589_2.png) [@SajjadAyoubi](https://discuss.huggingface.co/u/SajjadAyoubi)\
**Post date:** [September 27, 2021, 5:07pm UTC](https://discuss.huggingface.co/t/bigbirdpegasus-vs-bigbird/10340/1 "2021-09-27T17:07:00Z")

</div>

I’ve seen the BigBirdPegasus and BigBird pages on transformers doc but don’t understand the difference.  
BigBirdPegasus is like pretrained BigBird Encoder+pretrained BigBird Decoder then fine-tuned for summarization?