# Tokenization of Sequencepair for Pipeline

**URL:** <https://discuss.huggingface.co/t/tokenization-of-sequencepair-for-pipeline/6115>\
**Category:** Beginners\
**Created:** [May 11, 2021, 9:36am UTC](https://discuss.huggingface.co/t/tokenization-of-sequencepair-for-pipeline/6115 "2021-05-11T09:36:27Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![lewa](https://avatars.discourse-cdn.com/v4/letter/l/82dd89/32.png) [@lewa](https://discuss.huggingface.co/u/lewa)\
**Post date:** [May 11, 2021, 9:36am UTC](https://discuss.huggingface.co/t/tokenization-of-sequencepair-for-pipeline/6115/1 "2021-05-11T09:36:27Z")

</div>

Hi,

for my model I tokenize my input like this:

`encoded_dict = xlmr_tokenizer(term1, term2, max_length=max_len, padding='max_length',truncation=True, return_tensors='pt')`

so that it gets 2 different strings as input (term1, term2), that it separates with the special separator token.

How can I use the huggingface pipeline for input like this?  
If I load the pipeline like this:

```
tokenizer_xlmr = XLMRobertaTokenizerFast.from_pretrained("xlm-roberta-base")
my_pipeline = pipeline("sentiment-analysis", model=my_model, tokenizer=tokenizer_xlmr)
my_pipeline([term1,term2])

```

I get two separate predictions instead of one. Thanks in advance for any type of help!

---

<div class="post-metadata">

**Author:** ![lewa](https://avatars.discourse-cdn.com/v4/letter/l/82dd89/32.png) [@lewa](https://discuss.huggingface.co/u/lewa)\
**Post date:** [May 18, 2021, 4:34pm UTC](https://discuss.huggingface.co/t/tokenization-of-sequencepair-for-pipeline/6115/2 "2021-05-18T16:34:11Z")

</div>

Further information:  
calling  
`my_pipeline(term1+"</s></s>"+term2)`  
gives nearly the same output as using the trained model directly on the data from the encoded\_dict as defined above. The labels are the same, but the probability differ from the 5th/6th decimal place onwards. Why is that?