# How to Modifiy a model?

**URL:** <https://discuss.huggingface.co/t/how-to-modifiy-a-model/148832>\
**Category:** Beginners\
**Created:** [April 4, 2025, 9:30am UTC](https://discuss.huggingface.co/t/how-to-modifiy-a-model/148832 "2025-04-04T09:30:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Song222](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/song222/32/44765_2.png) [@Song222](https://discuss.huggingface.co/u/Song222)\
**Post date:** [April 4, 2025, 9:30am UTC](https://discuss.huggingface.co/t/how-to-modifiy-a-model/148832/1 "2025-04-04T09:30:54Z")

</div>

How should I modify the attention layer of the model and save the modified model? I want to know if this approach is feasible and whether there will be any problems（There are new parameters in my attention layer.）.

```
modeling_qwen2.QWEN2_ATTENTION_CLASSES["sdpa"] = MyQwen2Attn
modeling_qwen2.Qwen2SdpaAttention = MyQwen2Attn

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-0.5B-Instruct" )
model.save_pretrained("./model")

```

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [April 4, 2025, 11:04am UTC](https://discuss.huggingface.co/t/how-to-modifiy-a-model/148832/2 "2025-04-04T11:04:10Z")

</div>

Perhaps this one?

> **[Attention Interface](https://huggingface.co/docs/transformers/main/en/attention_interface)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.