# Qwen 'padding\_side = right' problem

**URL:** <https://discuss.huggingface.co/t/qwen-padding-side-right-problem/152318>\
**Category:** Models\
**Created:** [April 25, 2025, 3:42pm UTC](https://discuss.huggingface.co/t/qwen-padding-side-right-problem/152318 "2025-04-25T15:42:08Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rgb2gbr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rgb2gbr/32/46330_2.png) [@rgb2gbr](https://discuss.huggingface.co/u/rgb2gbr)\
**Post date:** [April 25, 2025, 3:42pm UTC](https://discuss.huggingface.co/t/qwen-padding-side-right-problem/152318/1 "2025-04-25T15:42:09Z")

</div>

You are attempting to perform batched generation with padding\_side=‘right’ this may lead to unexpected behaviour for Flash Attention version of Qwen2. Make sure to call `tokenizer.padding_side = 'left'` before tokenizing the input  
this is the problem although it set to left

Temp Solution(if “tokenizer.padding\_side = ‘left’” didn’t work ):  
=============\> find ‘transformers/models/qwen2/modeling\_qwen2.py’  
and change line ~622 or comment

```
    if self.config._attn_implementation == "flash_attention_2":
        if attention_mask is not None and past_key_values is not None:
            is_padding_right = attention_mask[:, -1].sum().item() != input_tensor.size()[0]
            if is_padding_right:
                print('I dont gaf')
                continue
                # raise ValueError(
                # "You are attempting to perform batched generation with padding_side='right'"
                # " this may lead to unexpected behaviour for Flash Attention version of Qwen2. Make sure to "
                # " call `tokenizer.padding_side = 'left'` before tokenizing the input. "
                # )
        if attention_mask is not None and 0.0 in attention_mask:
            return attention_mask
        return None

```

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [April 25, 2025, 4:04pm UTC](https://discuss.huggingface.co/t/qwen-padding-side-right-problem/152318/2 "2025-04-25T16:04:06Z")

</div>

Oh…

> <https://github.com/huggingface/transformers/issues/36100>
>
> \### System Info
> 
> !\[Image\](https://github.com/user-attachments/assets/6ecbc96d-d3…4a-4164-903a-0ef65ea65fb0)
> 
> !\[Image\](https://github.com/user-attachments/assets/e92f3446-3e81-4887-9f9c-0b5cb3047683)
> 
> !\[Image\](https://github.com/user-attachments/assets/8bef88c1-40ba-413b-8444-d018c9691787)
> the padding side should be left as qwen2 vl do .
> 
> \### Information
> 
> \- \[\] The official example scripts
> \- \[x\] My own modified scripts
> 
> \### Tasks
> 
> \- \[\] An officially supported task in the \`examples\` folder (such as GLUE/SQuAD, ...)
> \- \[x\] My own task or dataset (give details below)
> 
> \### Reproduction
> 
> run conditional generation using qwen2\_5\_vl using flash attention 2 .
> 
> \### Expected behavior
> 
> !\[Image\](https://github.com/user-attachments/assets/e92f3446-3e81-4887-9f9c-0b5cb3047683)

> You can set the padding side when loading the processor as `AutoProcessor.from_pretrained(model_id, padding_side="left")`. We usually recommend to make sure padding side is on the correct side, before generating 🙂  
> We might need to update the model docs, if example code for batch generation failed with the same error. Feel free to open a PR if so

---

<div class="post-metadata">

**Author:** ![rgb2gbr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rgb2gbr/32/46330_2.png) [@rgb2gbr](https://discuss.huggingface.co/u/rgb2gbr)\
**Post date:** [April 25, 2025, 4:31pm UTC](https://discuss.huggingface.co/t/qwen-padding-side-right-problem/152318/3 "2025-04-25T16:31:27Z")

</div>

it works on VL(Text2Image) version not regular version of Qwen