# What is the correct form of decoder\_input\_ids for LEDForConditionalGeneration?

**URL:** <https://discuss.huggingface.co/t/what-is-the-correct-form-of-decoder-input-ids-for-ledforconditionalgeneration/7947>\
**Category:** 🤗Transformers\
**Created:** [July 3, 2021, 5:03am UTC](https://discuss.huggingface.co/t/what-is-the-correct-form-of-decoder-input-ids-for-ledforconditionalgeneration/7947 "2021-07-03T05:03:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![user123123](https://avatars.discourse-cdn.com/v4/letter/u/858c86/32.png) [@user123123](https://discuss.huggingface.co/u/user123123)\
**Post date:** [July 3, 2021, 5:03am UTC](https://discuss.huggingface.co/t/what-is-the-correct-form-of-decoder-input-ids-for-ledforconditionalgeneration/7947/1 "2021-07-03T05:03:16Z")

</div>

I have been looking into the articles on the web, but unfortunately I cannot find the clear answer. I guess one of them is the correct `decoder_input_ids` (`label` should be decoder\_input\_ids[1:] ?):

```auto
1) <s>...</s><pad>...<pad>
2) </s><s>...</s><pad>...<pad>
3) <pad><pad>...</s><s>...</s>

```

Thanks in advance.

+) I am going to fine-tune this model for free form QA.

---

<div class="post-metadata">

**Author:** ![user123123](https://avatars.discourse-cdn.com/v4/letter/u/858c86/32.png) [@user123123](https://discuss.huggingface.co/u/user123123)\
**Post date:** [July 5, 2021, 2:32am UTC](https://discuss.huggingface.co/t/what-is-the-correct-form-of-decoder-input-ids-for-ledforconditionalgeneration/7947/2 "2021-07-05T02:32:32Z")

</div>

I guess `decoder_input_ids` should be `</s><s>...` (without `</s>`), given `label` as `<s>...</s>`, according to the code below used for generating `decoder_input_ids`:

```auto
def shift_tokens_right(input_ids: torch.Tensor, pad_token_id: int, decoder_start_token_id: int):
    """
    Shift input ids one token to the right.
    """
    shifted_input_ids = input_ids.new_zeros(input_ids.shape)
    shifted_input_ids[:, 1:] = input_ids[:, :-1].clone()
    shifted_input_ids[:, 0] = decoder_start_token_id

    assert pad_token_id is not None, "config.pad_token_id has to be defined."
    # replace possible -100 values in labels by `pad_token_id`
    shifted_input_ids.masked_fill_(shifted_input_ids == -100, pad_token_id)

    return shifted_input_ids

```