# Gemma-2 & Phi-3 SFT nuances

**URL:** <https://discuss.huggingface.co/t/gemma-2-phi-3-sft-nuances/107260>\
**Category:** Models\
**Created:** [September 18, 2024, 5:51pm UTC](https://discuss.huggingface.co/t/gemma-2-phi-3-sft-nuances/107260 "2024-09-18T17:51:12Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![preivey](https://avatars.discourse-cdn.com/v4/letter/p/a698b9/32.png) [@preivey](https://discuss.huggingface.co/u/preivey)\
**Post date:** [September 18, 2024, 5:51pm UTC](https://discuss.huggingface.co/t/gemma-2-phi-3-sft-nuances/107260/1 "2024-09-18T17:51:12Z")

</div>

There are a few minor nuances that are important during fine-tuing/inference of LLMs. I am experimenting with Gemma-2 and Phi-3 and here are the little things that I don’t quiet get:

1. Some LLM do not have pad token. Does it matter if i change set it as unknown or eos?

2. Does the padding size matter? [Here](https://huggingface.co/docs/transformers/llm_tutorial#wrong-padding-side) it states that " Since LLMs are not trained to continue from pad tokens, your input needs to be left-padded", which I do not get, because if we pad from the left we will have pad tokens then the text, so the model have to continue from pad. Also during fine-tuning with flash\_attention\_2 it throws a warning to put padding\_side=‘right’.

3. Does it makes sense to use in context learning after SFT?  
I will update the list.
