# \` transformers\` '4.57.0.dev0' is not compatible with \`evaluate\`?

**URL:** <https://discuss.huggingface.co/t/transformers-4-57-0-dev0-is-not-compatible-with-evaluate/168737>\
**Category:** 🤗Transformers\
**Created:** [September 28, 2025, 3:28am UTC](https://discuss.huggingface.co/t/transformers-4-57-0-dev0-is-not-compatible-with-evaluate/168737 "2025-09-28T03:28:17Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zoe0427](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/zoe0427/32/53729_2.png) [@Zoe0427](https://discuss.huggingface.co/u/Zoe0427)\
**Post date:** [September 28, 2025, 3:28am UTC](https://discuss.huggingface.co/t/transformers-4-57-0-dev0-is-not-compatible-with-evaluate/168737/1 "2025-09-28T03:28:17Z")

</div>

Hi, I try to use ` transformers` ‘4.57.0.dev0’, but when I import `evaluate'`：

File “”, line 241, in \_call\_with\_frames\_removed  
File “/home/–/dev\_env/lib/python3.10/site-packages/evaluate/ **init**.py”, line 29, in  
from .evaluation\_suite import EvaluationSuite  
File “/home/–/dev\_env/lib/python3.10/site-packages/evaluate/evaluation\_suite/ **init**.py”, line 10, in  
from ..evaluator import evaluator  
File “/home/–/dev\_env/lib/python3.10/site-packages/evaluate/evaluator/ **init**.py”, line 17, in  
from transformers.pipelines import SUPPORTED\_TASKS as SUPPORTED\_PIPELINE\_TASKS  
File “/home/–/transformers/src/transformers/pipelines/ **init**.py”, line 63, in  
from .document\_question\_answering import DocumentQuestionAnsweringPipeline  
File “/home/–/transformers/src/transformers/pipelines/document\_question\_answering.py”, line 30, in  
from .question\_answering import select\_starts\_ends  
File “/home/–/transformers/src/transformers/pipelines/question\_answering.py”, line 9, in  
from ..data import SquadExample, SquadFeatures, squad\_convert\_examples\_to\_features

---

<div class="post-metadata">

**Author:** ![JamesDavids](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jamesdavids/32/54347_2.png) [@JamesDavids](https://discuss.huggingface.co/u/JamesDavids)\
**Post date:** [September 28, 2025, 8:12am UTC](https://discuss.huggingface.co/t/transformers-4-57-0-dev0-is-not-compatible-with-evaluate/168737/2 "2025-09-28T08:12:56Z")

</div>

You’ve run into a **version mismatch problem** between `transformers` and `evaluate`.

Here’s what’s happening:

- You’re on a **dev version** of `transformers` (`4.57.0.dev0`).

- That version moved/removed some internals (like `SquadExample`, `SquadFeatures`, and `squad_convert_examples_to_features`).

- But your installed `evaluate` package still imports those old classes → import error.

* * *

### ✅ How to Fix

1. **Use matching stable versions**  
The easiest fix: install **stable releases** of both packages that are tested together. For example:

2. **If you must stay on dev branch** (`4.57.0.dev0`)

3. **Temporary workaround**  
If you only need evaluation metrics (BLEU, accuracy, etc.), you can skip the broken imports by using:

* * *

### ⚡ Recommendation

Unless you’re contributing to Hugging Face core dev, avoid the `dev0` builds — they’re unstable and will break like this. Stick with the latest PyPI release.

---

<div class="post-metadata">

**Author:** ![linguistique](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/linguistique/32/54393_2.png) [@linguistique](https://discuss.huggingface.co/u/linguistique)\
**Post date:** [September 30, 2025, 12:37am UTC](https://discuss.huggingface.co/t/transformers-4-57-0-dev0-is-not-compatible-with-evaluate/168737/3 "2025-09-30T00:37:12Z")

</div>

Make sure you select “always use the latest environment”