# Fine-tuning translator based on a single language

**URL:** <https://discuss.huggingface.co/t/fine-tuning-translator-based-on-a-single-language/10224>\
**Category:** Intermediate\
**Created:** [September 22, 2021, 3:56pm UTC](https://discuss.huggingface.co/t/fine-tuning-translator-based-on-a-single-language/10224 "2021-09-22T15:56:41Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![DenMarC](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/denmarc/32/2762_2.png) [@DenMarC](https://discuss.huggingface.co/u/DenMarC)\
**Post date:** [September 22, 2021, 3:56pm UTC](https://discuss.huggingface.co/t/fine-tuning-translator-based-on-a-single-language/10224/1 "2021-09-22T15:56:41Z")

</div>

Hi,

I have to develop a translator where there are models trained only on the source language. For example, EN\_KLINGON, where only EN\_DE, EN\_FR and EN\_RU are available. I was wondering if I could take advantage of the trained (EN) encoder, while training the decoder from zero. Also, since such an encoder could be biased towards the target language, could I use a base language model (BERT, for example) for the source language as its encoder in the Seq2Seq training pipeline?