Inconsitancies in model implementation
#24
by Samoed - opened
Hi, Qwen team!
It seems the implementation with SentenceTransformers is a bit different from what was reported. Previously in MTEB we had an implementation with your updates to the Transformers class, but later we decided to update it to use the SentenceTransformers version, but with it we got slightly different results than reported, while with the previous implementation we can reproduce them. You can see the full results comparison in this comment. This can be fixed by https://huggingface.co/Qwen/Qwen3-VL-Embedding-2B/discussions/22