# Pytorch Language Modeling Example for Seq2Seq Models

**URL:** <https://discuss.huggingface.co/t/pytorch-language-modeling-example-for-seq2seq-models/163333>\
**Category:** 🤗Transformers\
**Created:** [July 16, 2025, 7:17pm UTC](https://discuss.huggingface.co/t/pytorch-language-modeling-example-for-seq2seq-models/163333 "2025-07-16T19:17:27Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![hrezaei](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hrezaei/32/31681_2.png) [@hrezaei](https://discuss.huggingface.co/u/hrezaei)\
**Post date:** [July 16, 2025, 7:17pm UTC](https://discuss.huggingface.co/t/pytorch-language-modeling-example-for-seq2seq-models/163333/1 "2025-07-16T19:17:27Z")

</div>

I customized the run\_clm.py example to be able to pretrain seq2seq models like T5, hence named [run\_slm.py](https://github.com/HRezaei/transformers/commit/195ab5bc640f98efb82abe58c03030dad7453c84). I think it might save time for other people interested in something similar. If you agree, please let me know to create a PR. Thanks! 🤗

@ArthurZ