# Adding custom layer to GPT-2

**URL:** <https://discuss.huggingface.co/t/adding-custom-layer-to-gpt-2/23677>\
**Category:** Models\
**Created:** [September 27, 2022, 1:32pm UTC](https://discuss.huggingface.co/t/adding-custom-layer-to-gpt-2/23677 "2022-09-27T13:32:05Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vibhu04](https://avatars.discourse-cdn.com/v4/letter/v/c2a13f/32.png) [@Vibhu04](https://discuss.huggingface.co/u/Vibhu04)\
**Post date:** [September 27, 2022, 1:32pm UTC](https://discuss.huggingface.co/t/adding-custom-layer-to-gpt-2/23677/1 "2022-09-27T13:32:05Z")

</div>

Hi everyone,  
I am looking for a way to modify GPT-2’s architecture slightly by inserting a custom feedforward layer inside a GPT-2 decoder block right after the masked self-attention sublayer. Is there a way to achieve this using Hugging Face’s GPT-2 model? I’m new to Hugging Face, any suggestions would be appreciated.  
Thank you!