# SFT Trainer and chat templates **URL:** \ **Category:** Beginners\ **Created:** [March 24, 2025, 3:58pm UTC](https://discuss.huggingface.co/t/sft-trainer-and-chat-templates/147205 "2025-03-24T15:58:14Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![reubenrouse](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@reubenrouse](https://discuss.huggingface.co/u/reubenrouse)\ **Post date:** [March 24, 2025, 3:58pm UTC](https://discuss.huggingface.co/t/sft-trainer-and-chat-templates/147205/1 "2025-03-24T15:58:14Z")
Hello I’m implementing a framework for fine-tuning various LLMs using the TRL library’s SFTTrainer. I have a question about how chat templates work: 1. When using SFTTrainer with datasets in the standard formats (with “messages” array or “prompt”/“completion” fields), does the trainer automatically apply the tokenizer’s chat\_template? The documentation suggests it does. 2. For models whose tokenizers don’t have a chat\_template attribute set (or it’s empty), what template does SFTTrainer apply by default? Is it using ChatML format? 3. For maximum performance, should I always manually set the appropriate chat\_template on the tokenizer before passing it to SFTTrainer? ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [March 24, 2025, 5:05pm UTC](https://discuss.huggingface.co/t/sft-trainer-and-chat-templates/147205/2 "2025-03-24T17:05:03Z")
Just to be sure, I also asked Hugging Chat, and it seems to be okay. I think it probably works fairly well with the default settings. > > > Hi! I am interested in using the \`SFTTrainer\` for instruction-tuning. Following …\[the docs\](https://huggingface.co/docs/trl/main/en/sft\_trainer#dataset-format-support), I can see that I can provided examples in the following format to have the trainer format things for me: > > \`\`\`json > {"prompt": "\", "completion": "\"} > {"prompt": "\", "completion": "\"} > {"prompt": "\", "completion": "\"} > \`\`\` > > The docs also say: > > \> The \[SFTTrainer\](https://huggingface.co/docs/trl/main/en/trainer#trl.SFTTrainer) will then format the dataset for you using the defined format from the model’s tokenizer with the \[apply\_chat\_template\](https://huggingface.co/docs/transformers/main/en/chat\_templating#templates-for-chat-models) method. > > My question and confusion is, what does the trainer do if the tokenizer has no \`chat\_template\`, as is the case with the \[base llama model\](https://huggingface.co/meta-llama/Llama-2-13b-hf/blob/main/tokenizer\_config.json)? > **[How to fine-tune Google Gemma with ChatML and Hugging Face TRL](https://www.philschmid.de/fine-tune-google-gemma)** > > In this blog post you will learn how to fine tune Google Gemma using Hugging Face Transformers, Datasets and TRL. * * * The following is from Hugging Chat. When using the SFTTrainer in the TRL library with datasets in standard formats (such as those with a “messages” array or “prompt”/“completion” fields), the trainer does automatically apply the tokenizer’s chat\_template, according to the documentation [1][3][4]. This behavior is facilitated by the `apply_chat_template` method of the tokenizer, which is used to format the dataset into a structure suitable for training chat models. The setup is often handled using the `setup_chat_format` function from TRL, which configures the model and tokenizer with the necessary special tokens and formatting for conversational tasks [2][4]. If your dataset is in one of the supported standard formats, you can pass it directly to the SFTTrainer without pre-processing, and the trainer will handle the formatting for you [1][4]. When using the `SFTTrainer` and the tokenizer does not have a `chat_template` attribute set (or it is empty), the trainer does not automatically fall back to a default template like ChatML. Instead, the behavior depends on how the tokenizer is configured and whether you explicitly define a chat template for the model. If the tokenizer does not have a `chat_template` attribute, the `apply_chat_template` method will either raise an error or fail to format the input, as it relies on the template being defined to structure the conversations appropriately [2]. This is because different models may use different chat formats during pre-training, and the chat template must match the model’s expected input format for optimal performance [2]. If you want to use a specific template (e.g., ChatML), you should explicitly set it as the `chat_template` in the tokenizer before using `SFTTrainer`. For example, you can define a ChatML template as follows: ```python tokenizer.chat_template = "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}" ``` Once the `chat_template` is set, the `SFTTrainer` will use it to format the input data [2]. In summary: - If a `chat_template` is not set, the `SFTTrainer` will not automatically use a default template like ChatML. - You must explicitly define and set the `chat_template` for the tokenizer if one is not already provided. - If no template is defined, the `apply_chat_template` method will fail, as it requires a template to format the conversation data [2][1]. If you are training a model from scratch or fine-tuning it for chat, you have the flexibility to choose a template (e.g., ChatML) and configure it accordingly [2]. For maximum performance when using the SFTTrainer in the TRL library, it is highly recommended to manually set the appropriate `chat_template` on the tokenizer before initiating the training process. Here’s a structured overview of the reasoning and steps involved: ### Why Manually Set the `chat_template`? 1. **Consistency with Model Expectations** : Different models are pre-trained on specific chat formats. Setting the correct `chat_template` ensures that the input data aligns with the model’s expected format, enhancing training effectiveness. 2. **Special Tokens Handling** : Many chat templates, such as ChatML, include special tokens (e.g., `<|im_start|>`). Ensuring these tokens are correctly formatted helps the model recognize and process them during training. 3. **Avoiding Default Limitations** : Relying on default settings can lead to suboptimal results if the tokenizer’s default template does not match your specific use case or model requirements. ### Steps to Manually Set the `chat_template` 1. **Choose the Right Template** : Decide on the chat template format that best suits your model and task. Common formats include ChatML and Alpaca. 2. **Define the Template** : Create a Jinja template string that structures conversations. For instance, a ChatML template might look like: 3. **Apply the Template** : Use the `setup_chat_format` function from the TRL library to apply the template to both the model and tokenizer. 4. **Initialize SFTTrainer** : Pass the configured tokenizer and model to the SFTTrainer, ensuring the data collator and other parameters are set correctly. ### Conclusion Manually setting the `chat_template` is a crucial step for aligning the input data with the model’s expectations, especially for optimal performance in fine-tuning tasks. By defining the template explicitly, you ensure that the data is formatted correctly, include necessary special tokens, and thus maximize the effectiveness of the training process. ---
**Author:** ![reubenrouse](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@reubenrouse](https://discuss.huggingface.co/u/reubenrouse)\ **Post date:** [March 25, 2025, 1:50pm UTC](https://discuss.huggingface.co/t/sft-trainer-and-chat-templates/147205/3 "2025-03-25T13:50:43Z")
Thanks a lot man, this is really helpful ! ---
**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\ **Post date:** [March 26, 2025, 1:51am UTC](https://discuss.huggingface.co/t/sft-trainer-and-chat-templates/147205/4 "2025-03-26T01:51:08Z")
Closed on March 26, 2025, 1:51am UTC This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.