# Smaller embedding size causes lower loss

**URL:** <https://discuss.huggingface.co/t/smaller-embedding-size-causes-lower-loss/20755>\
**Category:** 🤗Transformers\
**Created:** [July 23, 2022, 7:30am UTC](https://discuss.huggingface.co/t/smaller-embedding-size-causes-lower-loss/20755 "2022-07-23T07:30:49Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![canovich](https://avatars.discourse-cdn.com/v4/letter/c/aeb1de/32.png) [@canovich](https://discuss.huggingface.co/u/canovich)\
**Post date:** [July 23, 2022, 7:30am UTC](https://discuss.huggingface.co/t/smaller-embedding-size-causes-lower-loss/20755/1 "2022-07-23T07:30:49Z")

</div>

When I convert my multilingual transformer model to a single lingual transformer model(got my languages embedding from the multilingual transformer and deleted other embeddings, decreased dimensions of embedding layers), the loss is much less. But I didn’t understand why. What can be the reason for that?