# Where does causal mask get generated for T5 decoder? **URL:** \ **Category:** Beginners\ **Created:** [January 9, 2024, 12:04am UTC](https://discuss.huggingface.co/t/where-does-causal-mask-get-generated-for-t5-decoder/68439 "2024-01-09T00:04:02Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![rranjan1](https://avatars.discourse-cdn.com/v4/letter/r/df788c/32.png) [@rranjan1](https://discuss.huggingface.co/u/rranjan1)\ **Post date:** [January 9, 2024, 12:04am UTC](https://discuss.huggingface.co/t/where-does-causal-mask-get-generated-for-t5-decoder/68439/1 "2024-01-09T00:04:02Z")
I am trying to figure out the causal mask implementation for T5 encoder-decoder model. The docstring for decoder\_attention\_mask says that “Causal mask will also be used by default”. However, I do not find any code in modeling\_t5.py that is generating the causal mask. Can somebody point me to the place where causal mask is generated for T5 decoder? ---
**Author:** ![rranjan1](https://avatars.discourse-cdn.com/v4/letter/r/df788c/32.png) [@rranjan1](https://discuss.huggingface.co/u/rranjan1)\ **Post date:** [January 9, 2024, 12:30am UTC](https://discuss.huggingface.co/t/where-does-causal-mask-get-generated-for-t5-decoder/68439/2 "2024-01-09T00:30:12Z")
Found it. The causal mask gets created in get\_extended\_attention\_mask here: [transformers/src/transformers/modeling\_utils.py at main · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/main/src/transformers/modeling_utils.py#L979) ---
**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\ **Post date:** [January 9, 2024, 12:30pm UTC](https://discuss.huggingface.co/t/where-does-causal-mask-get-generated-for-t5-decoder/68439/3 "2024-01-09T12:30:39Z")
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.