# Why this BERTScore has a high precision?

**URL:** <https://discuss.huggingface.co/t/why-this-bertscore-has-a-high-precision/175276>\
**Category:** 🤗Transformers\
**Created:** [April 15, 2026, 12:51pm UTC](https://discuss.huggingface.co/t/why-this-bertscore-has-a-high-precision/175276 "2026-04-15T12:51:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![IanCodez](https://avatars.discourse-cdn.com/v4/letter/i/b9bd4f/32.png) [@IanCodez](https://discuss.huggingface.co/u/IanCodez)\
**Post date:** [April 15, 2026, 12:51pm UTC](https://discuss.huggingface.co/t/why-this-bertscore-has-a-high-precision/175276/1 "2026-04-15T12:51:02Z")

</div>

Hello everyone, I’ve been studying transformer lately and I confused with BERTScore. So I tried to check the score of “who are you” and “look at the dogs” but I dont understand why the precision is really high? its preicions is 86% or am I doing it wrong?

Here is my code

```auto
bert_metric.compute(
    predictions=["who are you"],
    references=[["look at the dogs"]],
    lang='en'
)

```

and here is the output  
`{‘precision’: [0.8612378835678101],`  
`‘recall’: [0.8896267414093018],`  
`‘f1’: [0.8752021789550781],`  
`‘hashcode’: ‘roberta-large_L17_no-idf_version=0.3.12(hug_trans=5.0.0)’}`

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [April 16, 2026, 7:45am UTC](https://discuss.huggingface.co/t/why-this-bertscore-has-a-high-precision/175276/2 "2026-04-16T07:45:57Z")

</div>

It seems to be a feature of the library that the scores can be hard to interpret unless you explicitly rescale them…

* * *

No. You are not doing it wrong.

The confusing part is that **BERTScore precision is not “86% correct”**. BERTScore is a **token-matching similarity metric**. It compares each token in your prediction with tokens in the reference using **contextual embeddings** and **cosine similarity** , then computes precision, recall, and F1 from those matches. The official Hugging Face metric docs describe it exactly that way. ([GitHub](https://github.com/huggingface/evaluate/blob/main/metrics/bertscore/README.md "evaluate/metrics/bertscore/README.md at main"))

## What your result really means

Your output:

```python
{'precision': [0.8612],
 'recall': [0.8896],
 'f1': [0.8752],
 'hashcode': 'roberta-large_L17_no-idf_version=0.3.12(hug_trans=5.0.0)'}

```

does **not** mean:

- the two sentences are 86% semantically similar
- or that 86% of the words are “correct”

It means something narrower:

- for each token in **“who are you”** , BERTScore found the **best-matching token** in **“look at the dogs”**
- those best-match similarities were fairly high on average
- so the final precision came out around `0.86` ([GitHub](https://github.com/Tiiiger/bert_score "Tiiiger/bert\_score: BERT score for text generation"))

## Why this can happen even for unrelated sentences

BERTScore uses **soft matches** , not exact matches.

So it is not asking:

> “Are these two sentences obviously the same meaning?”

It is asking something closer to:

> “How similar are the candidate tokens to their best reference-token matches in embedding space?” ([GitHub](https://github.com/Tiiiger/bert_score "Tiiiger/bert\_score: BERT score for text generation"))

That is why a short unrelated pair can still score high.

## Why your score is especially high

Your hashcode says:

- `roberta-large`
- layer `L17`
- `no-idf`

That is the standard English setup. The official BERTScore repo uses **`roberta-large` as the default English model** , and its rescaling note says raw BERTScore values with large RoBERTa models often live in a **narrow high range** , often around **0.85 to 0.95**. So your `0.8612` is surprising to a human reader, but **not surprising to the metric**. ([GitHub](https://github.com/Tiiiger/bert_score "Tiiiger/bert\_score: BERT score for text generation"))

## Is your input format okay?

Yes.

This is valid:

```python
references=[["look at the dogs"]]

```

The Hugging Face BERTScore metric accepts `references` as either:

- a list of strings, or
- a list of lists of strings

The nested-list form is for **multiple references per prediction**. Since you only gave one reference, your format is fine. ([GitHub](https://github.com/huggingface/evaluate/blob/main/metrics/bertscore/README.md "evaluate/metrics/bertscore/README.md at main"))

## The easiest way to think about it

Imagine BERTScore doing this:

- candidate: `who | are | you`
- reference: `look | at | the | dogs`

For precision, it roughly does:

- best match for `who`
- best match for `are`
- best match for `you`
- average those 3 values

So if those best matches were something like:

```plaintext
who -> 0.84
are -> 0.87
you -> 0.87

```

then the average would be about `0.86`.

That still would **not** mean the sentences are 86% similar. It would only mean the **best token-level contextual matches** were high. ([Comet](https://www.comet.com/site/blog/bertscore-for-llm-evaluation/ "BERTScore For LLM Evaluation - Comet"))

## What to do if you want a more human-readable score

Use:

```python
rescale_with_baseline=True

```

The official BERTScore rescaling note says this was added because raw scores are hard to interpret and are often compressed into a small range. ([GitHub](https://github.com/Tiiiger/bert_score/blob/master/journal/rescale_baseline.md "bert\_score/journal/rescale\_baseline.md at master"))

Example:

```python
bert_metric.compute(
    predictions=["who are you"],
    references=["look at the dogs"],
    lang="en",
    rescale_with_baseline=True,
)

```

## If your real goal is sentence similarity

If what you really want is:

> “How semantically similar are these two sentences?”

then a sentence-similarity model is usually a better fit than raw BERTScore. Sentence-Transformers’ official docs are specifically about **Semantic Textual Similarity** for this kind of task. ([Sentence Transformers](https://sbert.net/docs/sentence_transformer/usage/semantic_textual_similarity.html "Semantic Textual Similarity"))

## Bottom line

The simplest answer is:

- **Your code is fine**
- **Your interpretation is the part to change**
- **`0.8612` is not “86% similar”**
- it is a **raw token-level contextual similarity score**
- and with default English `roberta-large`, raw BERTScore values often look **artificially high** to humans ([GitHub](https://github.com/Tiiiger/bert_score "Tiiiger/bert\_score: BERT score for text generation"))