# Tensor not on the same device after using BitsandBytesConfig

**URL:** <https://discuss.huggingface.co/t/tensor-not-on-the-same-device-after-using-bitsandbytesconfig/94313>\
**Category:** 🤗Transformers\
**Created:** [June 27, 2024, 2:55am UTC](https://discuss.huggingface.co/t/tensor-not-on-the-same-device-after-using-bitsandbytesconfig/94313 "2024-06-27T02:55:49Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![gfuiu](https://avatars.discourse-cdn.com/v4/letter/g/41988e/32.png) [@gfuiu](https://discuss.huggingface.co/u/gfuiu)\
**Post date:** [June 27, 2024, 2:55am UTC](https://discuss.huggingface.co/t/tensor-not-on-the-same-device-after-using-bitsandbytesconfig/94313/1 "2024-06-27T02:55:49Z")

</div>

Hello, I am a beginner to llm and I was trying to use Qlora to fine tune the llama3-7B model. I was dealing with a text classification problem, so I used AutoModelForSequenceClassification(model\_name, quantization\_config=bnb\_config, num\_labels=100), and I use deepspeed zero2 to reduce the memory used in each gpu. When I removed the quantization\_config in AutoModelForSequenceClassification, the code runs perfectly, but when I add this parameter, the error goes like: Expected all tensors on the same device. Does anyone know what is going on there? Thanks for your time!