# Returning score associated with prediction\_value from loaded\_tokenizer

**URL:** <https://discuss.huggingface.co/t/returning-score-associated-with-prediction-value-from-loaded-tokenizer/110994>\
**Category:** Beginners\
**Created:** [October 9, 2024, 4:33pm UTC](https://discuss.huggingface.co/t/returning-score-associated-with-prediction-value-from-loaded-tokenizer/110994 "2024-10-09T16:33:40Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![trevorwooddmu](https://avatars.discourse-cdn.com/v4/letter/t/7bcc69/32.png) [@trevorwooddmu](https://discuss.huggingface.co/u/trevorwooddmu)\
**Post date:** [October 9, 2024, 4:33pm UTC](https://discuss.huggingface.co/t/returning-score-associated-with-prediction-value-from-loaded-tokenizer/110994/1 "2024-10-09T16:33:40Z")

</div>

Hi. I’m trying to get the %age score corresponding to prediction\_value using the following python code snippet

```
predict_input = loaded_tokenizer.encode(email_text, truncation=True, padding=True, return_tensors="tf")
output = loaded_model(predict_input)[0]

prediction_value = tf.argmax(output, axis=1).numpy()[0]

```

loaded\_model and loaded\_tokenizer have been trained earlier using distilbert.

I’ve tried using TextClassificationPipeline, but unfortunately “email\_text” has too many tokens (more than 512) so doesn’t work. If I truncate email\_text it returns incorrect prediction\_value.

Thanks in advance

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [October 10, 2024, 4:32am UTC](https://discuss.huggingface.co/t/returning-score-associated-with-prediction-value-from-loaded-tokenizer/110994/2 "2024-10-10T04:32:34Z")

</div>

I searched to see if it was some kind of bug or configuration issue, but it seems to be a deep-rooted problem.  
By the way, you can pass options to the pipeline for the model and tokenizer in addition to the pipeline options. Check the model description to see what options are available. It would be easy to solve this problem.

> **[Papers with Code - Breaking the Token Barrier: Chunking and Convolution for...](https://paperswithcode.com/paper/breaking-the-token-barrier-chunking-and)**
>
> No code available yet.

> **[How to Classify Long Documents and Texts with BERT Models](https://www.mercity.ai/blog-post/classify-long-texts-with-bert)**
>
> Most of the applications need access to process long texts and documents, BERT models even though competent, might not perform very good for long contexts. In this blog, we show how to work with long texts and contexts with BERT.

> <https://github.com/huggingface/transformers/issues/15177>
>
> \# 🚀 Feature request
> 
> Currently, the token-classification pipeline truncates in…put texts longer than 512 tokens. 
> It would be great if the pipeline could process texts of any length.
> 
> \## Motivation
> 
> This issue is a result of this feature request:
> huggingface/hub-docs#11
> 
> As suggested I am tagging @Narsil here. 
> 
> \## Your contribution
> 
> For my \[punctuation prediction model\](https://huggingface.co/oliverguhr/fullstop-punctuation-multilang-large), I wrote an inference code for this task.
> This code segments a longer text into 512 tokens chunks with some overlap and 
> reconstructs the final output string. The code however is rather complex and tailored to 
> my punctuation prediction use case. So I am not sure if this is useful here.

---

<div class="post-metadata">

**Author:** ![trevorwooddmu](https://avatars.discourse-cdn.com/v4/letter/t/7bcc69/32.png) [@trevorwooddmu](https://discuss.huggingface.co/u/trevorwooddmu)\
**Post date:** [October 11, 2024, 11:39am UTC](https://discuss.huggingface.co/t/returning-score-associated-with-prediction-value-from-loaded-tokenizer/110994/3 "2024-10-11T11:39:24Z")

</div>

Thank you. It haddn’t occured to me that the input could be chunked. Although it’ll make the classification slower, if it makes it more accurate, that’ll be really helpful. I’ll look into it