# Inserting custom layer in GPT-2

**URL:** <https://discuss.huggingface.co/t/inserting-custom-layer-in-gpt-2/23665>\
**Category:** Beginners\
**Created:** [September 27, 2022, 8:42am UTC](https://discuss.huggingface.co/t/inserting-custom-layer-in-gpt-2/23665 "2022-09-27T08:42:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vibhu04](https://avatars.discourse-cdn.com/v4/letter/v/c2a13f/32.png) [@Vibhu04](https://discuss.huggingface.co/u/Vibhu04)\
**Post date:** [September 27, 2022, 8:42am UTC](https://discuss.huggingface.co/t/inserting-custom-layer-in-gpt-2/23665/1 "2022-09-27T08:42:43Z")

</div>

Hi everyone,  
I am looking for a way to modify GPT-2’s architecture slightly by inserting a custom feedforward layer inside a GPT-2 decoder block right after the masked self-attention sublayer. Is there a way to achieve this using Hugging Face’s GPT-2 model? I’m new to Hugging Face, any inputs would be appreciated.  
Thank you!